Unfortunately the Metop scripts don't work for AVHRRL1 data files from Internet. This is for V3.0, but as far as I can see also the case for v4.0.

These files have 90 minutes in one file; nearly 500M per file, after unzipping nearly 1 GByte!

The problem is with the method of preselecting files using tle, which is now done by using the position halfway each Metop data file. With 90-minute files, which is about one full orbit, this cannot work.

For the rest, Satpy works fine, so these files 'just' need another selection criterium. The MSG files from Internet do also work, but with a different reader.

Apart from that, unfortunately the AVHRR files cannot be processed on a Pi, they are just too big for even the biggest Pi (8Gbyte RAM). MSG from internet works fine on a Pi, and also AVHRR coming from Eumetcast.

Regards,
Rob

On Mon, Nov 21, 2022 at 12:45 PM, R. Alblas wrote:

The problem is with the method of preselecting files using tle, which is now done by using the position halfway each Metop data file. With 90-minute files, which is about one full orbit, this cannot work.

Rob,

the problem sounds similar to the Sentinel-3 OLCI ERR files that cover half an orbit. I have written a crude workaround for that here:

https://groups.io/g/MSG-1/message/33041

Cheers,
Ernst

Hello,

I'm hijacking this topic for a related subject.

I've always been curious how to do area/orbit/temporal selection properly, with LEO satellite granules. If you know of any kind of documentation on how to interpret and handle file names, please point me in the right direction. I want to try my hands on incorporating data processing available on https://www.earthdata.nasa.gov even though Python is not a very friendly language to me.

Regards,
George

On Tue, Nov 22, 2022 at 05:38 AM, George Sz wrote:

If you know of any kind of documentation on how to interpret and handle file names, please point me in the right direction.

George,

there is normally UTC start (DateTime) and stop (DateTime) of the granule in the file name.
Using TLEs of the satellite and knowing the swath of the sensor you find the area covered.
Inside the data there is very precise positional data for almost every pixel for reprojection.


Ernst

Thanks Ernst and Rob (over email) for the feedback.

I understand that I can use pyorbital for this purpose but I really have to think this through. The workflow would look like this, I presume?

1. use pyorbital to predict passes over my area of interest
2. cross-reference date ranges with the granules list (file names) downloaded previously
3. download granules of interest
4. process imagery

No. 2 looks the hardest to me right now.

George

George,

Why don't you have a look at the scripts of Ernst? it's all sorted out there; easier than inventing the wheel again, even if you are not that familiar with Python.

Regards,

Rob.

George,

to avoid reinventing the wheel you must have a look at

http://pytroll.github.io/

and especially PyTROLL/satpy

https://github.com/pytroll/satpy

or even polar2grid

https://github.com/ssec/polar2grid

first.

For the work flow with LEO granules I see two methods:

1) If you must download your files from an Internet source:
Use pyorbital to find previous passes of a satellite over your POI
a) then you can either download by time span if your source allows
b) or you must construct the exact filenames and download these
Most Internet sources will have a kind of API (that does a) and/or b)
on the server) and even provide documentation + software to use it.

2) If you have most/all available LEO files anyway (as with
EUMETCast where up to 0.5 TB of data fills our HDDs per day!):
a) go through the list of files of a LEO for a special period
b) read the start and stop times of the granules (segments)
c) use pyorbital just to pick out the files near your POI.
That's what I do in my PyTROLL/Satpy for EUMETCast packages.

Best regards,
Ernst