Date
1 - 11 of 11
SPS script for MTG-I1 stops with RecursionError
|
I'm trying to get SPS working with the MTG-I1 sample data currently being sent on HVS-2, but I'm having problems. Perhaps someone who has this working can advise?
I have a GOES-16-Carribean Python script which works, so at least some of the basics are correct. I have an MTI1.py script from Douglas (thanks) which I edited to point to my data, and to have isbulk = True and my correct receiver type. The files now look quite similar to my inexpert eye. I can run the script and the virtual memory consumed by Python gradually increases up to over 15 GB, at gradually returns to a more normal value, eventually producing the messages: File "C:\Tools\miniconda3\envs\pytroll\Lib\site-packages\satpy\readers\file_handlers.py", line 291, in available_datasets for is_avail, ds_info in (configured_datasets or []): File "C:\Tools\miniconda3\envs\pytroll\Lib\site-packages\satpy\readers\file_handlers.py", line 291, in available_datasets for is_avail, ds_info in (configured_datasets or []): [Previous line repeated 988 more times] RecursionError: maximum recursion depth exceeded Perhaps this is an obvious error? I've attached the MTI1.py file, and the output from the job, in one composite text file. Any help would be appreciated. Incidentally, EUMETSAT use the name: "MTG-I1". Any reason why this isn't used but "MTI1" is used instead? Many thanks, David -- SatSignal Software - Quality software for you Web: https://www.satsignal.eu Email: david-taylor@... Twitter: @gm8arv |
|
|
|
Ernst Lobsiger
David,
1) The naming convention of SPS 4.1 expects the file to be named MTI1-area.py to start with. Where 'area' should later be replaced by then name of the actual area chosen. 2) The SPS 4.1 distributed module GEOstuff.py contained a search line limited for the MTG download data available at that time. It has to be adapted as explained here: https://groups.io/g/MSG-1/message/34445 Regards, Ernst |
|
|
|
On 14/03/2023 10:41, Ernst Lobsiger via groups.io wrote:
David,Thanks, Ernst. 1. I had wondered where the area stuff came from and then I discovered the .yml file. I chose "isleofman" and the recursion error was fixed. 2. The glob fix resulted in "Sorry, no good files found". Working: if sat=='MTI1': files = glob(segdir + '/' + 'W_XX-EUMETSAT*.nc') return files Not working: if sat=='MTI1': files = glob(segdir + '/' + 'W_XX-EUMETSAT*MTI1+FCI-1C*_GTT_DEV_'+Dat[:-1]+'*.nc') return files (Wretched line breaks!) The file names I'm seeing are like: W_XX-EUMETSAT-Darmstadt,IMG+SAT,MTI1+FCI-1C-RRAD-FDHSI-FD--CHK-BODY---NC4E_C_EUMT_20170920092411_GTT_DEV_20170920091849_20170920091911_N_JLS_T_0056_0038.nc My Python doesn't yet stretch to "Dat[:-1]", but it looks OK otherwise. Command line: python MTI1-isleofman.py 201709201100 Cheers, David -- SatSignal Software - Quality software for you Web: https://www.satsignal.eu Email: david-taylor@... Twitter: @gm8arv |
|
|
|
On 14/03/2023 10:41, Ernst Lobsiger via groups.io wrote:
David,[] Regards,Ernst, Now getting "IndexError: index 0 is out of bounds for axis 0 with size 0" at the end of a run. Please see the attached.... Cheers, David -- SatSignal Software - Quality software for you Web: https://www.satsignal.eu Email: david-taylor@... Twitter: @gm8arv |
|
|
|
Ernst Lobsiger
On Tue, Mar 14, 2023 at 04:37 AM, David J Taylor GM8ARV 🏴 🇪🇺 wrote:
Not working:David, it should work as above and works here under GNU/Linux. The point is that for a slot like 201709201200 you will get file names with minutes 1201, 1202, ...up to maybe 1209. That's why I glob this slot for 120* . Maybe you asked for a slot not yet disseminated today? Graham and Douglas might comment on Windows. Ernst |
|
|
|
Ernst Lobsiger
On Tue, Mar 14, 2023 at 04:56 AM, Ernst Lobsiger wrote:Maybe you asked for a slot not yet disseminated today?(pytroll)
David, This slot seems not yet available ... eumetcast@kallisto:~/SPStools/DEVscripts$ python MTI1-area.py 201709201100 Sorry, no good files found ... But I just made an image with 201709200900 Ernst |
|
|
|
Ernst Lobsiger
On Tue, Mar 14, 2023 at 05:07 AM, Ernst Lobsiger wrote:
This slot seems not yet available ...David and All, I think that explains what we see: Originally the MTI1 test dissemination was planned to end 2023/02/27 (according to a comment of mine). Then apparently EUMETSAT prolonged this test period to 2023 March 6th: https://www.eumetsat.int/simulated-mtg-fci-l1c-test-dataset-dissemination But as a matter of Murphy they stopped today 09:35 UTC. These are the latest MTI1 files I got from today. If your system deletes these old files from 2017 at midnight (as mine does), you might only be able to image the slots up to 2017/09/20/09:20 UTC (and maybe tomorrow all the MTI1 test files will be gone). Cheers, Ernst |
|
|
|
On 14/03/2023 13:07, Ernst Lobsiger via groups.io wrote:
David and All,OK, Ernst, that makes sense. I rapidly marked the files I have as read-only in the hope that I can keep them for a little. Better put them in a safe directory too! So the remaining issues: 1 - Error report says that it can't handle the "TRAIL" files - I guess these are like the old "epilog" files detailing what happened during the pass. Obviously found from the wildcard. 2 - A couple of invalid data messages - see the attached log file. 3 - How do I add my own location? I see Edinburgh in the cities.txt file but it's not displayed. Thanks, David -- SatSignal Software - Quality software for you Web: https://www.satsignal.eu Email: david-taylor@... Twitter: @gm8arv |
|
|
|
Ernst Lobsiger
David,
yes the TRAIL file (number 41) is something like the MSG EPI files. Apparently it's not used/understood by the current MTG satpy reader. python is very noisy. You will always see a couple of error or warning messages. Your location can either be entered for all scripts in GEOstuff.py as "station". You can also define a cities list per script. If this list is let empty, then a default list (per satellite) defined in GEOstuff.py applies. You can also edit this default list and enter or add 'Edinburgh' there. Ernst |
|
|
|
On 14/03/2023 15:28, Ernst Lobsiger via groups.io wrote:
David,Ernst, Understood about TRAIL - I guess will get fixed in due course. If not, I might be tempted to look at it. I always try to make my stuff compile, and certainly run, without errors or warnings. I think that's at least polite to the users! If there are messages, I like them to be easy to understand and to indicate clearly to the user what has to be done. Cities: added in GEOstuff.py, but it's in a different colour so that needs to be fixed at some point. One thin I did do was to enable the caching. Anyone coming to this for the first time will despair of the time taken. With caching enabled it's much quicker - highly recommended. Thanks for your help and patience. Cheers, David -- SatSignal Software - Quality software for you Web: https://www.satsignal.eu Email: david-taylor@... Twitter: @gm8arv |
|
|
|
Ernst Lobsiger
On Tue, Mar 14, 2023 at 08:46 AM, David J Taylor GM8ARV 🏴 🇪🇺 wrote:
Understood about TRAIL - I guess will get fixed in due course. If not, I might be tempted to look at it.David, you can get rid of the TRAIL file message by: files = glob(segdir + '/' + 'W_XX-EUMETSAT*MTI1+FCI-1C-RRAD-FDHSI-FD--CHK-BODY-*_GTT_DEV_' + Dat[:-1] + '*.nc') It's not yet clear, whether the reader will be updated to get some information from the TRAIL file. That's why I let as is. For the rest of unwanted messages (as long as everything works), just redirect those to 2>NUL. I know you will have a hard time with that, but that's just how it is if a lot of different packages and coders are part of a big thing like PyTROLL/Satpy. On the other hand I experienced that most users have no clue whatever error messages say in a very understandable English. Ernst |
|
|