Dear All,

here comes my "Satpy Processing System" (SPS) V 4.1.
While V 4.0 was labeled as "Beta", I hope V 4.1 now
deserves the name "Stable". Sorry for distributing
this once more with wetransfer. I wanted to put V 4.1
on github.com but realized that this is not the right
place to distribute binaries and images. Documentation
has been enhanced and new LEO satellites Sen3A/B (ERR)
added. There are now also combined pairs of satellites
Sen3X, NOAAX and MetopX possible. Users that invested
a lot of work for V 4.0 can try to just exchange the modules
LEOstuff.py and GEOstuff.py (experiment in DEVscripts!).
Just pick out from the *.zip and *.tgz what ever you need.


Get SPS V 4.1 for GNU/Linux here (app. 780MB, *.tgz):

https://we.tl/t-fHtOlk2Jk3


Get SPS V 4.1 for Windows 10/11 here (app. 780MB, *.zip:

https://we.tl/t-BNm80gGvli


Unfortunately RSS is currently down so that the latest code
changes could not be tested with Meteosat-10 *.webm movies.

Douglas should find an easy workaround for *.eps-glb files.
Graham will certainly try MetopX-africa1km.py. Yes, that's
what you can possibly do now if you have 128GB of RAM ;-\.

Many thanks to the Windows/Linux "Beta" testers of V 4.0.

Have fun,
Ernst

P.S.
Future versions (2023?) might be slimmed down to 100MB
so that they even fit into the MSG-1 file section. But
then users will have to get some big stuff like GSHHG
and ffmpeg.exe again from the Internet to make it fly.

On 11/12/2022 08:47, Ernst Lobsiger via groups.io wrote:

Get SPS V 4.1 for GNU/Linux here (app. 780MB, *.tgz):
https://we.tl/t-fHtOlk2Jk3 <https://we.tl/t-fHtOlk2Jk3>
Get SPS V 4.1 for Windows 10/11 here (app. 780MB, *.zip:
https://we.tl/t-BNm80gGvli <https://we.tl/t-BNm80gGvli>

Thanks for that, Ernst.

When clicked, the site says there are two files to download (for each transfer) but only one file comes down. Both files here are .Zip

Linux: 849 MB
Windows: 798 MB

It would be helpful to have an MD5 signature for these files to help ensure a correct download. I would like to add this information to the Wiki, but could you put the files on a more permanent site? GoogleDrive, DropBox, Microsoft OneDrive or whatever?

Your effort in producing these is really appreciated. I hope to try the software thinking primarily of MTG data as the actual use, but learning with MSG data.

Cheers,
David
--
SatSignal Software - Quality software for you
Web: https://www.satsignal.eu
Email: david-taylor@...
Twitter: @gm8arv

On Sun, Dec 11, 2022 at 01:20 AM, David J Taylor GM8ARV 🏴󠁧󠁢󠁳󠁣󠁴󠁿 🇪🇺 wrote:

It would be helpful to have an MD5 signature for these files to help ensure a correct download.

David,

sorry I have prepared hashes here but forgot to attach them.

Ernst

On Sun, Dec 11, 2022 at 01:20 AM, David J Taylor GM8ARV 🏴󠁧󠁢󠁳󠁣󠁴󠁿 🇪🇺 wrote:

When clicked, the site says there are two files to download (for each transfer) but only one file comes down. Both files here are .Zip

David,

apparently wetransfer zips the files once more. So my two *.zip and two *.tgz are inside this  *.zip you get from wetransfer.
The SHA256 sums just published should be checked with my files inside (unfortunately most users will not do that though).

Best regards,
Ernst

On Sun, Dec 11, 2022 at 01:20 AM, David J Taylor GM8ARV 🏴󠁧󠁢󠁳󠁣󠁴󠁿 🇪🇺 wrote:

I hope to try the software thinking primarily of MTG data as the actual use, but learning with MSG data.

David,

regarding MTG-I1 there is a test script inside but this expects
at least one slot of the latest EUMETSAT test data to be available
locally. There is still very active Satpy development for MTG-I1
(notably from people at EUMETSAT) going on. I also have scripts
for Metop 2nd gen. working with test data from EUMETSAT. But let's
hope for a perfect launch of MTG and first real data in early 2023.

This SPS package as is more a motivation pack where I included a
couple of images to show how it works and what it can do. I will
probably slim it down to the bare software part in the future.

TBH I have none of these cloud drives that Microsoft and other
big companies always want me to open and fill with personal data.

Regards,
Ernst

On 11/12/2022 08:47, Ernst Lobsiger via groups.io wrote:

Dear All,
here comes my "Satpy Processing System" (SPS) V 4.1.
While V 4.0 was labeled as "Beta", I hope V 4.1 now
deserves the name "Stable". Sorry for distributing
this once more with wetransfer. I wanted to put V 4.1
on github.com but realized that this is not the right
place to distribute binaries and images. Documentation
has been enhanced and new LEO satellites Sen3A/B (ERR)
added. There are now also combined pairs of satellites
Sen3X, NOAAX and MetopX possible. Users that invested
a lot of work for V 4.0 can try to just exchange the modules
LEOstuff.py and GEOstuff.py (experiment in DEVscripts!).
Just pick out from the *.zip and *.tgz what ever you need.
Get SPS V 4.1 for GNU/Linux here (app. 780MB, *.tgz):
https://we.tl/t-fHtOlk2Jk3 <https://we.tl/t-fHtOlk2Jk3>
Get SPS V 4.1 for Windows 10/11 here (app. 780MB, *.zip:
https://we.tl/t-BNm80gGvli <https://we.tl/t-BNm80gGvli>
Unfortunately RSS is currently down so that the latest code
changes could not be tested with Meteosat-10 *.webm movies.
Douglas should find an easy workaround for *.eps-glb files.
Graham will certainly try MetopX-africa1km.py. Yes, that's
what you can possibly do now if you have 128GB of RAM ;-\.
Many thanks to the Windows/Linux "Beta" testers of V 4.0.
Have fun,
Ernst
P.S.
Future versions (2023?) might be slimmed down to 100MB
so that they even fit into the MSG-1 file section. But
then users will have to get some big stuff like GSHHG
and ffmpeg.exe again from the Internet to make it fly.

=======================================================================

Many thanks again Ernst for all your hard work with this.
Looking forward to integrating it with V4.0.
At the moment upgrading my satellite system having recently added a new mini-computer so my progress may be a little slower this time whilst I set that up.

Kind regards,
Douglas.

Dear all,

The "Satpy Processing System" V 4.1 is now more permanently available from my web-site:

http://www.alblas.demon.nl/SatPy/index.html

These are "bare" versions (created by Ernst), that is, without generated example images and __pycache__ subdirs. Saves a lot of download time (total size less than half the size of the compressed wetransfer files).

Regards,

Rob.

Hi Ernst

many thanks for all the work you have put into producing your Version 4.1 Satpy set of tools
I have got all the latest changes integrated into my scripts and its all working perfectly

Cant wait to try the new MTG data too

Kind Regards

Graham

On 13/12/2022 10:04, Graham Woolf wrote:

Hi Ernst
many thanks for all the work you have put into producing your Version 4.1 Satpy set of tools
I have got all the latest changes integrated into my scripts and its all working perfectly
Cant wait to try the new MTG data too
Kind Regards
Graham

===================================================================================

Yes I will second that Graham. Got a lot up and running now with 4.1 and just slowly adding further things in. Will soon have RSS back so that will be good.

Again many thanks Ernst.

Best regards,
Douglas.

On Tue, Dec 13, 2022 at 02:04 AM, Graham Woolf wrote:

Cant wait to try the new MTG data too

Graham,

the MTI1 script as is has only been tested with one SLOT of TEST data that I have downloaded.
As you remember this does not yet work with enhancements due to solar zenith angles missing.

EUMETSAT and DWD are still actively working in perfecting the Satpy reader and composites.
As soon as we have real data I will have to make some updates for module GEOstuff.py only.

Kind regards,
Ernst

Dear All,

with all the excitement of the launch of MTG-I1 yesterday I completely
missed the change from Himawari-8 to Himawari-9 data. As far as I can
see there is no immediate action to take as both birds use the same
filenames and both are positioned at 140.7°E according to OSCAR.

Himawari-9 images will still be saved and named like Himawari-8. I
propose to let that for now as a clean change to Himawari-9 means not
only renaming the scripts but a couple of changes in GEOstuff.py in
the directory structure and the Windows and GNU/Linux shell scripts.
SPS users are of course free to make these changes as a good exercise.

I noted that yesterday I received no Himawari data from 04:50-08:00 UTC.
Appart from that everything works as before, webm movies are generated.

Best regards,
Ernst

P.S.
GOES17 is still in SPS 4.1 though it will be replaced by GOES18 in
early January 2023. Different from Himawari the numbers 17 and 18
are part of the respective file names. It's probably precautious to
keep GOES17 in SPS for a while in the case of a unplanned fall back.

On 14/12/2022 12:15, Ernst Lobsiger via groups.io wrote:

Dear All,
with all the excitement of the launch of MTG-I1 yesterday I completely
missed the change from Himawari-8 to Himawari-9 data. As far as I can
see there is no immediate action to take as both birds use the same
filenames and both are positioned at 140.7°E according to OSCAR.
Himawari-9 images will still be saved and named like Himawari-8. I
propose to let that for now as a clean change to Himawari-9 means not
only renaming the scripts but a couple of changes in GEOstuff.py in
the directory structure and the Windows and GNU/Linux shell scripts.
SPS users are of course free to make these changes as a good exercise.
I noted that yesterday I received no Himawari data from 04:50-08:00 UTC.
Appart from that everything works as before, webm movies are generated.
Best regards,
Ernst
P.S.
GOES17 is still in SPS 4.1 though it will be replaced by GOES18 in
early January 2023. Different from Himawari the numbers 17 and 18
are part of the respective file names. It's probably precautious to
keep GOES17 in SPS for a while in the case of a unplanned fall back.

=======================================================================================

Good idea Ernst. There is plenty of other distractions at this time of the year anyway !
Everything so far is working perfectly.

Regards,
Douglas.

Ernst,

And now for something completely different:

A small error in your GEOstuff.py:

sys.exit('Satellite short name must be in', sat_list)
must be
sys.exit('Satellite short name must be in'+ str(sat_list))

On Sat, Dec 17, 2022 at 01:53 PM, R. Alblas wrote:

And now for something completely different:

Rob,

you probably found that adding HIMA9?! Thanks a lot.
Currently checking stuff at the Transport Stream level.

Ernst

All,

I'm currently working to improve Windows webm.cmd as I finally found a way to extract with
PowerShell frames of certain periods similar to what I can do with "find" under GNU/Linux.

And I found a typo that may exist in a couple of Windows webm.cmd files floating around.

# Move the movie when complete

# is the beginning of a GNU/Linux BASH comment and should of course in CMD files read:

:: Move the movie when complete

This exists 2x here and does just make CMD complain 2x "command # not found".
Windows users of SPS 4.1 that find these 2 typos can easily fix those with an editor.

Regards,
Ernst

On Tue, Jan 24, 2023 at 01:07 PM, Manu wrote:

I also noticed that tidy.cmd does not delete video files.

Manu,

...
:: Integer numbers given to subroutines mean: Delete file if older than N days !
...
tidy.cmd must be run as administrator because it uses program "forfiles".
As distributed with SPS 4.1 videos are deleted after 30 days. See below:
...
:: Delete GEO frames/images/movies
call :GEOtrim MSG2   3 10 30
...
You probably do not want to keep videos this long, so lower the 3rd number.
Or do you say you have *.webm videos older than 30 days that are not deleted?

Regards,
Ernst


...

On Tue, Jan 24, 2023 at 01:30 PM, Manu wrote:

Out of curiosity, who does the number(s) before correspond to?

Manu,

:: Delete GEO frames/images/movies
call :GEOtrim MSG2   3 10 30

I cannot make it even more obvious.
Ernst

Thanks Ernst, it's very clear now.


Manu