Date   
EUMETSAT User Workshop and Training on Fire Monitoring

David J Taylor GM8ARV 🏴󠁧󠁢󠁳󠁣󠁴󠁿 🇪🇺
 

There's a User Workshop and Training on Fire Monitoring which may interest you.

"An online event through interactive data discovery and user practices on state-of-the-art operational datasets for detection of fire, related emissions and impacts. Jointly organized by EUMETSAT, CAMS-ECMWF, AC SAF, LSA SAF with support from Copernicus."

Details:
https://training.eumetsat.int/course/view.php?id=360

Cheers,
David
--
SatSignal Software - Quality software for you
Web: https://www.satsignal.eu
Email: david-taylor@...
Twitter: @gm8arv

Re: Pytroll/SatPy for EUMETCast

Ernst Lobsiger
 

On Thu, May 7, 2020 at 06:26 AM, Graham Woolf wrote:
Hi Ernst

I dont have an immpac folder anywhere on my C drive

Kind Regards

Graham
Graham

immpac is just my windows user name. Look where you installed miniconda3 !

C:\Users\graha\miniconda3\...

I think. You should remember your user name should you?


Ernst

Re: Pytroll/SatPy for EUMETCast

Graham Woolf
 

Hi Ernst

I dont have an immpac folder anywhere on my C drive

Kind Regards

Graham

Re: Pytroll/SatPy for EUMETCast

Ernst Lobsiger
 

Graham

You find all possible composites for all readers in.../miniconda3/.... /satpy/etc

C:\Users\immpac\miniconda3\envs\pytroll\Lib\site-packages\satpy\etc\composites

You may have to adapt the list of decompressed files for SEVIRI images in my scripts.

You find all the instrument specs here (EUMETCast normally does not send you all)

C:\Users\immpac\miniconda3\envs\pytroll\Lib\site-packages\satpy\etc\readers


Happy hacking

Ernst


P.S. This kind of stuff is for the advanced Windows Pytroll/Satpy HOWTO 2.0 (written by someone else...)

TP 1 data rate.

geojohnt@...
 

Hello All,

Does anyone know the current total and maximum possible data rate, that TP 1 can disseminate?

I saw a couple of bursts of + 74 Mbps yesterday around 12:19 UTC.

Also, my instances of 2-3 second 'bursts' of negative Mbps with the SR1 dials seems to be increasing.
Or perhaps I'm just spending too much time at the computer?

Regards,
John.

 

Re: Pytroll/SatPy for EUMETCast

Graham Woolf
 

Hi Ernst

I cant seem to find an airmass composite for the MSG satellite

I know there should be one because EUMETSAT quite often show the airmass immage on their website for example here

https://www.eumetsat.int/website/home/Images/ImageLibrary/DAT_4943581.html

I know we have an airmass composite for GOES16 so it would be good if I could find one for Meteosat

Kind Regards

Graham

Re: Pytroll/SatPy for EUMETCast

Graham Woolf
 

Hi Ernst

I seem to have all the files

I have uploaded them to here

https://drive.google.com/open?id=1OYLfUg1fdSAMUg2RN09v1e3wJopsytlp

I wonder if you could download them and see if you get the same problem

Kind Regards

Graham

Re: Pytroll/SatPy for EUMETCast

Ernst Lobsiger
 

Hi Graham

I'm quite sure now there is something wrong with your data. On my primery receiver chain
Linux --NFS--> Linux I have the FY-3D data from yesterday. I tried the daylight pass for
Switzerland and Isle of Man 20200506. It's the same orbit Nr. 12824 so the images look
the same for both POIs.  I add a sorted (!) list of the FY-3D files you should have.

You can produce this list by the following changes:

In the middle of the script:

# Sorting seems not necessary?!
# bestfiles = sorted(bestfiles)
-->
# Sorting seems not necessary?!
bestfiles = sorted(bestfiles)

At the bottom of the script:

# This is just to printout nicely
# for name in bestfiles:
#    print (name)
-->
# This is just to printout nicely
for name in bestfiles:
   print (name)

It seems your process crashes before you reach the end. So you have to copy the printout below
the sorting line above. Then you call the script: python FY-3D_VIS.py 20200506 > bestfiles.txt

Compare your list bestfiles.txt with what I attached.

Cheers
Ernst

@Douglas: Don't worry. That kind of debuging is not what you must expect on a dayly basis ...

Re: Pytroll/SatPy for EUMETCast

Ernst Lobsiger
 

Hi Graham and Jeskynar

Below the (unsorted) list of files used for the daylight pass of FY-3D over Switzerland I showed before:
...
return func(*(_execute_task(a, cache) for a in args))
Z:/E1H-RDS-2/2020/05/05\FY3D_20200505_124000_124100_12810_MERSI_1000M_L1B.HDF
Z:/E1H-RDS-2/2020/05/05\FY3D_20200505_124400_124500_12811_MERSI_1000M_L1B.HDF
Z:/E1H-RDS-2/2020/05/05\FY3D_20200505_123600_123700_12811_MERSI_GEO1K_L1B.HDF
Z:/E1H-RDS-2/2020/05/05\FY3D_20200505_123500_123600_12810_MERSI_GEO1K_L1B.HDF
Z:/E1H-RDS-2/2020/05/05\FY3D_20200505_124100_124200_12811_MERSI_GEO1K_L1B.HDF
Z:/E1H-RDS-2/2020/05/05\FY3D_20200505_124200_124300_12811_MERSI_1000M_L1B.HDF
Z:/E1H-RDS-2/2020/05/05\FY3D_20200505_124200_124300_12811_MERSI_GEO1K_L1B.HDF
Z:/E1H-RDS-2/2020/05/05\FY3D_20200505_124300_124400_12811_MERSI_1000M_L1B.HDF
Z:/E1H-RDS-2/2020/05/05\FY3D_20200505_123700_123800_12810_MERSI_1000M_L1B.HDF
Z:/E1H-RDS-2/2020/05/05\FY3D_20200505_124500_124600_12811_MERSI_GEO1K_L1B.HDF
Z:/E1H-RDS-2/2020/05/05\FY3D_20200505_123800_123900_12810_MERSI_GEO1K_L1B.HDF
Z:/E1H-RDS-2/2020/05/05\FY3D_20200505_123800_123900_12810_MERSI_1000M_L1B.HDF
Z:/E1H-RDS-2/2020/05/05\FY3D_20200505_124100_124200_12811_MERSI_1000M_L1B.HDF
Z:/E1H-RDS-2/2020/05/05\FY3D_20200505_124000_124100_12810_MERSI_GEO1K_L1B.HDF
Z:/E1H-RDS-2/2020/05/05\FY3D_20200505_124500_124600_12811_MERSI_1000M_L1B.HDF
Z:/E1H-RDS-2/2020/05/05\FY3D_20200505_123600_123700_12811_MERSI_1000M_L1B.HDF
Z:/E1H-RDS-2/2020/05/05\FY3D_20200505_124300_124400_12811_MERSI_GEO1K_L1B.HDF
Z:/E1H-RDS-2/2020/05/05\FY3D_20200505_124400_124500_12811_MERSI_GEO1K_L1B.HDF
Z:/E1H-RDS-2/2020/05/05\FY3D_20200505_123900_124000_12811_MERSI_GEO1K_L1B.HDF
Z:/E1H-RDS-2/2020/05/05\FY3D_20200505_123700_123800_12810_MERSI_GEO1K_L1B.HDF
Z:/E1H-RDS-2/2020/05/05\FY3D_20200505_123900_124000_12811_MERSI_1000M_L1B.HDF
Z:/E1H-RDS-2/2020/05/05\FY3D_20200505_123500_123600_12810_MERSI_1000M_L1B.HDF

(pytroll) C:\EMCtools\pyscripts>
...
Quite something! I do not know what happens if a couple of those files are missing. Another possibility
maybe that EUMETCast has sent all files and you have properly received them but some channel data
is marked invalid inside. Just try other dates and POI positions to see whether the problem persists.

Best Regards
Ernst

Re: Pytroll/SatPy for EUMETCast

Jeskynar
 

On Wed, 6 May 2020 at 22:31, Graham Woolf <woolfg@...> wrote:
I am having trouble with the FY3D script
It runs OK but never seems to produce any data - I have checked and I do have data files for the last two days
This is the output I am getting

The following datasets were not created and may require resampling to be generated: DatasetID(name='true_color', wavelength=None, resolution=None, polarization=None, calibration=None, level=None, modifiers=None)
 
 "No dataset matching 'DatasetID(name='true_color', wavelength=None, resolution=None, polarization=None, calibration=None, level=None, modifiers=None)' found"

The message is often given when there aren't enough of the right files for the requested output, i.e. not all of red,green,blue to make true_color so check there are no missing files 

Re: Pytroll/SatPy for EUMETCast

Ernst Lobsiger
 

Graham

No problem here except for I do not have the data from yesterday (my receiver went down).
'true_color' is certainly an acceptable composite (see the attached composites mersi-2 file).

Are your datasets damaged? If you uncomment the two debug lines does it tell you something?

DEBUG OFF
# Why to hell is it not working?
# from satpy.utils import debug_on
# debug_on()

DEBUG ON
# Why to hell is it not working?
from satpy.utils import debug_on
debug_on()

Good Luck
Ernst

Re: Pytroll/SatPy for EUMETCast

Graham Woolf
 

Hi Ernst

I am having trouble with the FY3D script

It runs OK but never seems to produce any data - I have checked and I do have data files for the last two days

This is the output I am getting


(pytroll) C:\WINDOWS\system32>python T:\PythonScripts\WindowsScripts\FY-3D_VIS.py 20200506
The following datasets were not created and may require resampling to be generated: DatasetID(name='true_color', wavelength=None, resolution=None, polarization=None, calibration=None, level=None, modifiers=None)
Traceback (most recent call last):
  File "c:\Users\graha\miniconda3\envs\pytroll\lib\site-packages\satpy\readers\__init__.py", line 301, in __getitem__
    return super(DatasetDict, self).__getitem__(item)
KeyError: 'true_color'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "T:\PythonScripts\WindowsScripts\FY-3D_VIS.py", line 160, in <module>
    scn.save_dataset(composite,imgdir+'/FY-3D-Orbit_'+str(orbmax)+'-'+
  File "c:\Users\graha\miniconda3\envs\pytroll\lib\site-packages\satpy\scene.py", line 1302, in save_dataset
    return writer.save_dataset(self[dataset_id],
  File "c:\Users\graha\miniconda3\envs\pytroll\lib\site-packages\satpy\scene.py", line 679, in __getitem__
    return self.datasets[key]
  File "c:\Users\graha\miniconda3\envs\pytroll\lib\site-packages\satpy\readers\__init__.py", line 303, in __getitem__
    key = self.get_key(item)
  File "c:\Users\graha\miniconda3\envs\pytroll\lib\site-packages\satpy\readers\__init__.py", line 290, in get_key
    return get_key(match_key, self.keys(), num_results=num_results,
  File "c:\Users\graha\miniconda3\envs\pytroll\lib\site-packages\satpy\readers\__init__.py", line 245, in get_key
    raise KeyError("No dataset matching '{}' found".format(str(key)))
KeyError: "No dataset matching 'DatasetID(name='true_color', wavelength=None, resolution=None, polarization=None, calibration=None, level=None, modifiers=None)' found"


It doesnt mean much to me but it may to you

Kind Regards

Graham

Re: Pytroll/SatPy for EUMETCast

Douglas Deans
 

Douglas
Just download the zipfile attached to message:
https://groups.io/g/MSG-1/message/29367
Maybe the attachment was to big for your e-mailer
Ernst
==================================================

Thanks Ernst. I found it on the Groups io page.
Some busy days ahead I expect for a programming novice but I appreciate the effort you have put into this.

Regards,
Douglas.

Re: Pytroll/SatPy for EUMETCast

Ernst Lobsiger
 

On Wed, May 6, 2020 at 09:15 AM, Douglas Deans wrote:
On 06/05/2020 16:52, Christian Peters via groups.io wrote:
Ernst,

really nice package and a lot of work!
Thanks for providing this, I hope someone will start to use Python and
Pytroll/Satpy.
It's a lot of fun...even if it's sometimes hard to fiddle with problems.

Welcome back to GNU/Linux!

Regards,

Christian
====================================================================

Christian when was Ernst's e-mail sent. I have not received it nor is it
in my ISP's spam folder.

Regards,
Douglas.
Douglas

Just download the zipfile attached to message:

https://groups.io/g/MSG-1/message/29367

Maybe the attachment was to big for your e-mailer

Ernst

Re: Pytroll/SatPy for EUMETCast

Douglas Deans
 

On 06/05/2020 16:52, Christian Peters via groups.io wrote:
Ernst,
really nice package and a lot of work!
Thanks for providing this, I hope someone will start to use Python and Pytroll/Satpy.
It's a lot of fun...even if it's sometimes hard to fiddle with problems.
Welcome back to GNU/Linux!
Regards,
Christian
====================================================================

Christian when was Ernst's e-mail sent. I have not received it nor is it in my ISP's spam folder.

Regards,
Douglas.

Re: Pytroll/SatPy for EUMETCast

Christian Peters
 

Ernst,

really nice package and a lot of work!
Thanks for providing this, I hope someone will start to use Python and Pytroll/Satpy.
It's a lot of fun...even if it's sometimes hard to fiddle with problems.

Welcome back to GNU/Linux! 

Regards,

Christian

Am 05.05.20 um 15:06 schrieb Ernst Lobsiger via groups.io:

Dear All

Here comes the Pytroll/Satpy EUMETCast Windows Starter Kit as promised. It should be "Childs Play" to get it working.
I have double checked all the scripts on my Windows 10 Pro 64Bit PC that got the files readonly from a SAMBA share Z:.

My definition of "Childs Play" (for Douglas):

 0) You know that you have Windows 10 64Bit
 1) You are not completley new to EUMETCast
 2) You read and follow the HOWTO inside (RTFM)
 3) You know where you find your satellite files

The xyRITDecompress.exe might be a temporary solution. There has been no reaction from EUMETSAT so far.
Maybe Hugo can provide a far better Windows solution that does not depend on any additional *.dll file.
This Starter Kit provides scripts for 14 satellites. You might not receive them all with your station.
Pytroll/Satpy will probably be one of the first packages to process MTG files. You better warm up now.

@Graham Woolf: As you cracked that and will crack that very fast you could also give Douglas a hand?
@Christian Peters: Now I'll go right back to GNU/Linux. MC is far easier than this Windows 10 Explorer!

Good Luck
Ernst

P.S Please also report SUCCESS in ths thread :-) ...

Re: Pytroll/SatPy for EUMETCast

Graham Woolf
 

Hi Ernst

Thank you so much for that

I will have a look tomorrow as I am busy this afternoon

Hopefully they will be pretty similar to the files I have working at the moment

I will try my best to help but I am just as much a python novice as most I would think

Kind Regards

Graham

Re: Pytroll/SatPy for EUMETCast

Ernst Lobsiger
 

Dear All

Here comes the Pytroll/Satpy EUMETCast Windows Starter Kit as promised. It should be "Childs Play" to get it working.
I have double checked all the scripts on my Windows 10 Pro 64Bit PC that got the files readonly from a SAMBA share Z:.

My definition of "Childs Play" (for Douglas):

 0) You know that you have Windows 10 64Bit
 1) You are not completley new to EUMETCast
 2) You read and follow the HOWTO inside (RTFM)
 3) You know where you find your satellite files

The xyRITDecompress.exe might be a temporary solution. There has been no reaction from EUMETSAT so far.
Maybe Hugo can provide a far better Windows solution that does not depend on any additional *.dll file.
This Starter Kit provides scripts for 14 satellites. You might not receive them all with your station.
Pytroll/Satpy will probably be one of the first packages to process MTG files. You better warm up now.

@Graham Woolf: As you cracked that and will crack that very fast you could also give Douglas a hand?
@Christian Peters: Now I'll go right back to GNU/Linux. MC is far easier than this Windows 10 Explorer!

Good Luck
Ernst

P.S Please also report SUCCESS in ths thread :-) ...

Re: Pytroll/SatPy for EUMETCast

Hugo
 

Hi Ernst and Rob,

I compiled my EUMETCastView and PublicDecompWT_2.06 under Windows with a version of Mingw-w64 a long time ago and I used that library ever since.
For the moment I don't have access to the Windows 7 computer where I used to compile my windows versions of EUMETCastView. (I am for the moment in a different place ...).
I'am using a Windows 10 computer now,  but I need to reinstall everything to recompile EUMETCastView and  PublicDecompWT_2.06 .  I know that I made some changes to the code of  PublicDecompWT_2.06 , just to be able to compile it.
As soon as I get back to that Windows 7 computer I will publish the complete code of EUMETCastView  (and   PublicDecompWT_2.06)  as a Qt project on Github.

Kind regards,

Hugo 

Re: Pytroll/SatPy for EUMETCast

Ernst Lobsiger
 

Rob,

you are talking about mingw. I tried different versions of mingw under GNU/Linux and Cygwin but compilation of xRITDecompress always failed.
With mingw I have no clue anyway. Hugo might be of help here. I finally compiled with the latest versions of Cygwin32 (on Windows 7 32Bit) and
Cygwin64 (on Windows 10 64Bit) with nothing but binutils, gcc-g++ compiler and make added to the defaults. There was no mingw whatsoever.

Regards,
Ernst