Ernst,

Now that you've mentioned swap, have you tried zram by any chance? Maybe my idea is far-fetched but perhaps all that uncompressed image data could be squeezed quite a bit, lowering memory requirement somewhat, and trading some CPU for RAM of course. I've got to test this myself but now, I need to sleep.

George

Hello,

here is a benchmark using one of my Celsius W520 Power workstations
(Intel Xeon 3.30GHz, 24GB RAM, 2013) with PyTROLL/SatPy running only.
This works with just about not using any space on the swap partition.

It takes alternating Metop-B/C (GDS) passes to image full continents
and make four composites 'overview', 'natural_color', 'natural_enh',
'cloud_phase_distinction' (times include decompression of *.bz2 data):

AFRICA, area africa_2km, resolution 2km, width x height = 4975 x 4638
real    5m21.391s
user    17m50.442s
sys     1m25.146s


NORTH AMERICA, area namerica2km, resolution 2km, width x height = 4612 x 4498
real    6m31.953s
user    18m51.142s
sys     1m34.900s


SOUTH AMERICA, area samerica2km, resolution 2km, width x height = 3000 x 4000
real    5m27.978s
user    14m27.160s
sys     1m3.538s


So it looks like an Orange-Pi with 32GB RAM and GNU/Linux can do that as well.
Reading their site there seems to be limited Orange-Pi GNU/Linux support yet.

Full resolution images of all composites available here (wetransfer link works 1 week):

https://we.tl/t-V0afwQ2dWv

@George: Report when you have PyTROLL/SatPy running on an Orange-Pi 32GB RAM.

Cheers,
Ernst

Looks interesting George. I measured on my PC (not the Pi) a usage of max. about 10 GByte during processing of the 1 GByte files of AVHRR in native format. But note that AVHRR as such is not a problem for a 8 GB Pi, if you use the 28MB files from Eumetcast! (Didn't test that yet with multi-passes and multi-satellites, though.)

Regards,

Rob.

Hello,

I have seen a few posts mentioning the RPi for processing some satellite imagery, however there may be RAM constraints, even with the 8GB version. I'm checking the market now and then and it seems that newer SBCs keep popping up.

There is a new Orange Pi on the way that will have a 32GB version which I think should be plenty. I've only seen the 16GB one for preorder and the price doesn't seem too bad either. Its CPU should be a lot faster than that of a RPi4, which is a bonus. I understand OS/driver support may not be great but for processing purposes it should fare well, I think.

http://www.orangepi.org/html/hardWare/computerAndMicrocontrollers/details/Orange-Pi-5.html

I expect this list to expand and we may just get an ARM board suitable for processing most Eumetcast (and other) data at a great power budget!

Regards,
George