Pytroll/SatPy for EUMETCast..Help


maurizio.calvitti@...
 

 

Dear,

I am an old Eumetcast user, grown using 100% David's software, plus Eumetcastview to enjoy BAS, HVS-1 and HVS-2 products.

I am quite satisfied, but I know that there is always room for improvement.

I have been reading to you for a while now, and I see a lot of interest around Pytroll /SatPy.

I confess that I don't even know it a little.

Does this processing method offer additional benefits?

I'd like to try but I have no idea how to get started. Does it work on Windows or just Linux?

Can anyone help me on how to start this new experiment?

 

Thanks a lot

 

Regards,

 

Maurizio

 

_._,_._,_




Questo messaggio e i suoi allegati sono indirizzati esclusivamente alle persone indicate e la casella di posta elettronica da cui è stata inviata è da qualificarsi quale strumento aziendale.

La diffusione, copia o qualsiasi altra azione derivante dalla conoscenza di queste informazioni sono rigorosamente vietate (art. 616 c.p, D.Lgs. n. 196/2003 s.m.i. e GDPR Regolamento - UE 2016/679).

Qualora abbiate ricevuto questo documento per errore siete cortesemente pregati di darne immediata comunicazione al mittente e di provvedere alla sua distruzione. Grazie.

This e-mail and any attachments is confidential and may contain privileged information intended for the addressee(s) only.

Dissemination, copying, printing or use by anybody else is unauthorised (art. 616 c.p, D.Lgs. n. 196/2003 and subsequent amendments and GDPR UE 2016/679).

If you are not the intended recipient, please delete this message and any attachments and advise the sender by return e-mail. Thanks.



Ernst Lobsiger
 

Maurizio,

Pytroll/Satpy is in Python and works on GNU/Linux 64bit and Windows 10 64bit. Start here:

https://groups.io/g/MSG-1/message/29367

Meanwhile we use much more sophisticated scripts. But I think it's still a good starting point.

Regards,
Ernst


maurizio.calvitti@...
 

Thank you Ernst,

I start to have a look.

 

maurizio

 

Da: MSG-1@groups.io <MSG-1@groups.io> Per conto di Ernst Lobsiger via groups.io
Inviato: venerdì 11 dicembre 2020 19:47
A: MSG-1@groups.io
Oggetto: Re: [MSG-1] Pytroll/SatPy for EUMETCast..Help

 

Maurizio,

Pytroll/Satpy is in Python and works on GNU/Linux 64bit and Windows 10 64bit. Start here:

https://groups.io/g/MSG-1/message/29367

Meanwhile we use much more sophisticated scripts. But I think it's still a good starting point.

Regards,
Ernst


David J Taylor GM8ARV 🏴󠁧󠁢󠁳󠁣󠁴󠁿 🇪🇺
 

On 11/12/2020 18:46, Ernst Lobsiger via groups.io wrote:
Maurizio,
Pytroll/Satpy is in Python and works on GNU/Linux 64bit and Windows 10 64bit. Start here:
https://groups.io/g/MSG-1/message/29367 <https://groups.io/g/MSG-1/message/29367>
Meanwhile we use much more sophisticated scripts. But I think it's still a good starting point.
Regards,
Ernst
I'd love to see an article in the GEO Quarterly as an introduction to this process. It could be split into several parts so as not to over-burden the author, the editor, or the reader. Les does an excellent job with both the pictures and the words.

Would there be any sense in making an installable package for Windows, or does the software (as a collection of packages) not really suit that? A simple "double-click setup.exe" is what many people would prefer. It's the results they want, not the "under the hood".

I'm quite willing to help with the English (not that's likely to be required) or other proof-reading.

Thanks,
David
--
SatSignal Software - Quality software for you
Web: https://www.satsignal.eu
Email: david-taylor@blueyonder.co.uk
Twitter: @gm8arv


Ernst Lobsiger
 

On Fri, Dec 11, 2020 at 11:26 AM, David J Taylor GM8ARV 🏴󠁧󠁢󠁳󠁣󠁴󠁿 🇪🇺 wrote:
A simple "double-click setup.exe" is what many people would prefer. It's the results they want, not the "under the hood".
David

Pytroll/Satpy will *never* be that way. It's scripting, working at a prompt and using an editor then and when.
People that just want results and are reluctant to read documentation and even some code should not touch
Pytroll/Satpy. It's learning by doing: One week ago I had no idea about the SAR on Sentinel-1A/B. Of course
as a Pysicist that has worked in remote sensing I knew what SAR means even if had forgotten a couple of
technical details. Today we had the first 50x50m resolution SAR images of iceberg A68. And Ferdinand
already added an image of Larsen-C. That's the sort of things that can only be done with Pytroll/Satpy ...

Cheers,
Ernst


maurizio.calvitti@...
 

Ernst,

you are right.

In order to exploit the potential of Pytroll / Satpy most of us (used to working with a double-click configuration to start and go ..) have to reset their mindset.

However from what I understand you can appreciate the advantages of Pytroll / Satpy especially working with SAR on Sentinel 1A / B, while for the other Eumetcast services (Bas, HVS-1 and HVS2), the classic applications provided by David, plus Eumetcastviewer , they do their best.

It's right?

However I love challenges ...

I installed Pytroll / Satpy on my PC ... everything was fine ...

But now, if I want to process raw msg4 files what should I do? (Sorry my brain has been formatted by Windows over the years!).

 

Reading the basic guide I cannot start a process ...

Thank you very much

 

Ciao,

Maurizio

 

 

 

Da: MSG-1@groups.io <MSG-1@groups.io> Per conto di Ernst Lobsiger via groups.io
Inviato: venerdì 11 dicembre 2020 21:48
A: MSG-1@groups.io
Oggetto: Re: [MSG-1] Pytroll/SatPy for EUMETCast..Help

 

On Fri, Dec 11, 2020 at 11:26 AM, David J Taylor GM8ARV 🏴󠁧󠁢󠁳󠁣󠁴󠁿 🇪🇺 wrote:

A simple "double-click setup.exe" is what many people would prefer. It's the results they want, not the "under the hood".

David

Pytroll/Satpy will *never* be that way. It's scripting, working at a prompt and using an editor then and when.
People that just want results and are reluctant to read documentation and even some code should not touch
Pytroll/Satpy. It's learning by doing: One week ago I had no idea about the SAR on Sentinel-1A/B. Of course
as a Pysicist that has worked in remote sensing I knew what SAR means even if had forgotten a couple of
technical details. Today we had the first 50x50m resolution SAR images of iceberg A68. And Ferdinand
already added an image of Larsen-C. That's the sort of things that can only be done with Pytroll/Satpy ...

Cheers,
Ernst


David J Taylor GM8ARV 🏴󠁧󠁢󠁳󠁣󠁴󠁿 🇪🇺
 

On 11/12/2020 20:47, Ernst Lobsiger via groups.io wrote:
On Fri, Dec 11, 2020 at 11:26 AM, David J Taylor GM8ARV 🏴󠁧󠁢󠁳󠁣󠁴󠁿 🇪🇺 wrote:
A simple "double-click setup.exe" is what many people would prefer.
It's the results they want, not the "under the hood".
David
Pytroll/Satpy will *never* be that way. It's scripting, working at a prompt and using an editor then and when.
People that just want results and are reluctant to read documentation and even some code should not touch
Pytroll/Satpy. It's learning by doing: One week ago I had no idea about the SAR on Sentinel-1A/B. Of course
as a Pysicist that has worked in remote sensing I knew what SAR means even if had forgotten a couple of
technical details. Today we had the first 50x50m resolution SAR images of iceberg A68. And Ferdinand
already added an image of Larsen-C. That's the sort of things that can only be done with Pytroll/Satpy ...
Cheers,
Ernst
Installing the basic tools and perhaps a sample script should not be impossible. That's all I'm suggesting. Give people a helping hand.

Cheers,
David
--
SatSignal Software - Quality software for you
Web: https://www.satsignal.eu
Email: david-taylor@blueyonder.co.uk
Twitter: @gm8arv


Ernst Lobsiger
 

On Fri, Dec 11, 2020 at 11:57 PM, <maurizio.calvitti@...> wrote:

But now, if I want to process raw msg4 files what should I do? (Sorry my brain has been formatted by Windows over the years!).

 

Reading the basic guide I cannot start a process ...

Thank you very much

Maurizio,

all I can and will repeat is: Read the HOWTO. I know with Microsoft and especially with the advent of the internet reading has become an ancient art. But it's all explained in the HOWTO and you have to RTFM.
Assuming "raw msg4 files" is what you get from EUMETCast you simply go to ..EMCtools\pyscripts. There is a script msg4.py. Open it with Windows notepad (editor) and tel it where to find your files.
Start your (pytroll) environment and off you go at the miniconda3 prompt with "python msg4.py your_data_and_time" ...


Regards,
Ernst

P.S. If "raw msg4 files" means somthing else ask on the Pytroll/Satpy google list.


Ernst Lobsiger
 

On Sat, Dec 12, 2020 at 01:14 AM, David J Taylor GM8ARV 🏴󠁧󠁢󠁳󠁣󠁴󠁿 🇪🇺 wrote:
Installing the basic tools and perhaps a sample script should not be impossible. That's all I'm suggesting. Give people a helping hand.

Cheers,
David
David,

my "Pytroll_Satpy_EMC_4_Windows_Starter_Kit" IS this "helping hand" and it comes with 27 sample scripts for all sorts of sats and problems. You cannot expect more from a guy that normally prefers to do everything under GNU/Linux

Cheers,
Ernst


Graham Woolf
 

Hi

I have to say Ernst has gone above and beyond to help not just me but anybody who wants to get into Pytroll and Satpy

As he says his starter kit is very easy to follow and you should be creating images within a couple of hours - I cant see what else you would need

All that needs changing in his scripts is to replace the Linux data paths with your own Windows data paths.

All the sample files are very well documented and if you have any issues then someone in this group will be able to help

All my image scripts are automated  using the Task Scheduler so I dont really have to do any manual intervention

Regards

Graham


maurizio.calvitti@...
 

Ernst,

Your package it’s OK for me. Just the time to try and learn.  

 

Step by step…:

 

I have edited my msg4.py script telling  whereraw Eumetcast files come:        I:/Eumetcast/received/bas/EUMETSAT_Data_Channel_2/YYYYmmDDHHMM”

 

I  Started  "Anaconda Prompt (miniconda3)" from the menue (a black CMD Window  

Opened  with a prompt "(base) C:\users\username>" = Done

 

I ACTIVATED   pytroll by entering "conda activate pytroll"

My prompt is now "(pytroll) C:\users\username>" = done

 

 

I Changed to where I have my scripts with "cd C:\EMCtools\pyscripts".  

 

Now I see…..

(pytroll)  C:\EMCtools\pyscripts>

 

It is now that I  start my script by editing “python msg4.py YYYYmmDDHHMM”……..Right?

 

But the reply is  that python is not an executable command…

 

Why?

 

Many thank for your patience, but starting from yesterday I made progress.

 

Maurizio

 

 

 

 

Da: MSG-1@groups.io <MSG-1@groups.io> Per conto di Ernst Lobsiger via groups.io
Inviato: sabato 12 dicembre 2020 11:11
A: MSG-1@groups.io
Oggetto: Re: [MSG-1] Pytroll/SatPy for EUMETCast..Help

 

On Fri, Dec 11, 2020 at 11:57 PM, <maurizio.calvitti@...> wrote:

But now, if I want to process raw msg4 files what should I do? (Sorry my brain has been formatted by Windows over the years!).

 

Reading the basic guide I cannot start a process ...

Thank you very much

Maurizio,

all I can and will repeat is: Read the HOWTO. I know with Microsoft and especially with the advent of the internet reading has become an ancient art. But it's all explained in the HOWTO and you have to RTFM.
Assuming "raw msg4 files" is what you get from EUMETCast you simply go to ..EMCtools\pyscripts. There is a script msg4.py. Open it with Windows notepad (editor) and tel it where to find your files.
Start your (pytroll) environment and off you go at the miniconda3 prompt with "python msg4.py your_data_and_time" ...


Regards,
Ernst

P.S. If "raw msg4 files" means somthing else ask on the Pytroll/Satpy google list.


Ernst Lobsiger
 

Maurizio,

 >  I:/Eumetcast/received/bas/EUMETSAT_Data_Channel_2/YYYYmmDDHHMM”

This directory in the script should probably read (if your files are really there):

 I:/Eumetcast/received/bas/EUMETSAT_Data_Channel_2”

 
Start the script with real dates e.g. this morning:   python msg4.py 202012111000

Good luck
Ernst

 

 


Ernst Lobsiger
 

Maurizio,

If you use some kind of David's data managers the files might already be in a data structure as I use it under GNU/Linux. Then try in the script:

segdir = "I:/Eumetcast/received/bas/EUMETSAT_Data_Channel_2” + "/" + Yea + "/" + Mon + "/" + Day

which is  allmost what I distributed.

Cheers,
Ernst


maurizio.calvitti@...
 

 

 

Ernst, as you can see raw msg4 files are in the right directory

 

I edited msg4.py 

 

# Edit the 3 directories below according to your file layout

# My files are in a /MountPoint/Channel/YYYY/mm/DD structure

segdir = "I:\EUMETCast\received\bas\EUMETSAT_Data_Channel_2\YYYYmmDDHHMM"

tmpdir = "D:\EMCdata\tmpdirs\xmsg4"

imgdir = "D:\EMCdata\images\Meteosat-11"

 

 

But the result is not good….

 

 

Python not found……

if I try with  the YYYYmmDDHHMM is the same result…

What I missing?

 

Thanks..

 

Maurizio

 

 

 

 

 

 

Da: MSG-1@groups.io <MSG-1@groups.io> Per conto di Ernst Lobsiger via groups.io
Inviato: sabato 12 dicembre 2020 15:11
A: MSG-1@groups.io
Oggetto: Re: [MSG-1] Pytroll/SatPy for EUMETCast..Help

 

Maurizio,

 >  I:/Eumetcast/received/bas/EUMETSAT_Data_Channel_2/YYYYmmDDHHMM”

This directory in the script should probably read (if your files are really there):

 I:/Eumetcast/received/bas/EUMETSAT_Data_Channel_2”

 
Start the script with real dates e.g. this morning:   python msg4.py 202012111000

Good luck
Ernst

 

 


Graham Woolf
 

Maurizio

Create a windows batch file in the directory containing you script

Edit the batch file and put the three lines below in the file changing the paths to suit your system

Save the batch file and then double click on it to run

Thats how I run all my scripts


*******************************************************************************************************
call c:\users\graha\miniconda3\Scripts\activate pytroll
python T:\PythonScripts\WindowsScripts\NOAA20\Noaa_20_VIS.py 20201212
call C:\Users\graha\miniconda3\condabin\conda.bat deactivate

********************************************************************************************************

Regards

Graham


Ferdinand Valk
 

Maurizio,

 

It looks as if your installation itself went wrong. After activating pytroll you should be able to already call just ‘python’. This doesn’t happen. I suggest you reinstall following accurately the instructions from Ernst’s Howto. Your system response clearly indicates that Python is absent.

 

Ferdinand

 

From: MSG-1@groups.io On Behalf Of Graham Woolf
Sent: Saturday, 12 December, 2020 16:40
To: MSG-1@groups.io
Subject: Re: [MSG-1] Pytroll/SatPy for EUMETCast..Help

 

Maurizio

Create a windows batch file in the directory containing you script

Edit the batch file and put the three lines below in the file changing the paths to suit your system

Save the batch file and then double click on it to run

Thats how I run all my scripts

*******************************************************************************************************
call c:\users\graha\miniconda3\Scripts\activate pytroll
python T:\PythonScripts\WindowsScripts\NOAA20\Noaa_20_VIS.py 20201212
call C:\Users\graha\miniconda3\condabin\conda.bat deactivate

********************************************************************************************************

Regards

Graham


Ernst Lobsiger
 

On Sat, Dec 12, 2020 at 08:25 AM, <maurizio.calvitti@...> wrote:
segdir = "I:\EUMETCast\received\bas\EUMETSAT_Data_Channel_2\YYYYmmDDHHMM"
Maurizio,

Use:
segdir = "I:/Eumetcast/received/bas/EUMETSAT_Data_Channel_2”

tmpdir = "D:/EMCdata/tmpdirs/xmsg4"

imgdir = "D:/EMCdata/images/Meteosat-11"

And *YES*, even if you are in Windows use FORWARD "/" slashes.

If this still does not work and Windows complains that python is not executable your miniconda3 install is not good.
But after all you can open the miniconda prompt. At the prompt (base) or (pytroll) try just python and press enter.
You should start the Python interpreter and fall on a prompt >>>. From there you can go back with quit().

If Windows still complains and cannot execute python, is it possible that:

a) You do not have Windows 10 64 Bit ?
b) You have Windows 10 64 Bit but installed a Miniconda 32Bit version?
c) That you also did some sort of Python install from this silly Microsoft store?

Regards,
Ernst




Ernst Lobsiger
 

... or could it be:
...
d) That you installed Miniconda3 as administrator and want to use it later as unpriviledged user?
...
Ernst


David J Taylor GM8ARV 🏴󠁧󠁢󠁳󠁣󠁴󠁿 🇪🇺
 

On 12/12/2020 11:25, Ernst Lobsiger via groups.io wrote:
David,
my "Pytroll_Satpy_EMC_4_Windows_Starter_Kit" IS this "helping hand" and it comes with 27 sample scripts for all sorts of sats and problems. You cannot expect more from a guy that normally prefers to do everything under GNU/Linux
Cheers,
Ernst
That's great, Ernst.

Where can that script be found? Perhaps it could be added to the group's Files area?

I'm happy to help with an article describing how to use this, with a view to seeing it in the GEO Quarterly journal. These things can get lost when they exist just as message numbers!

Cheers,
David
--
SatSignal Software - Quality software for you
Web: https://www.satsignal.eu
Email: david-taylor@blueyonder.co.uk
Twitter: @gm8arv


Ernst Lobsiger
 

On Sat, Dec 12, 2020 at 09:22 AM, David J Taylor GM8ARV 🏴󠁧󠁢󠁳󠁣󠁴󠁿 🇪🇺 wrote:
Where can that script be found? Perhaps it could be added to the group's Files area?
David,

its here (publication maybe wait for V2.0):

https://groups.io/g/MSG-1/message/29367

I don't know if the file area is really used.

Regards,
Ernst