Date   

Re: Pytroll/SatPy for EUMETCast

Ernst Lobsiger
 

Graham

No problem here except for I do not have the data from yesterday (my receiver went down).
'true_color' is certainly an acceptable composite (see the attached composites mersi-2 file).

Are your datasets damaged? If you uncomment the two debug lines does it tell you something?

DEBUG OFF
# Why to hell is it not working?
# from satpy.utils import debug_on
# debug_on()

DEBUG ON
# Why to hell is it not working?
from satpy.utils import debug_on
debug_on()

Good Luck
Ernst


Re: Pytroll/SatPy for EUMETCast

Graham Woolf
 

Hi Ernst

I am having trouble with the FY3D script

It runs OK but never seems to produce any data - I have checked and I do have data files for the last two days

This is the output I am getting


(pytroll) C:\WINDOWS\system32>python T:\PythonScripts\WindowsScripts\FY-3D_VIS.py 20200506
The following datasets were not created and may require resampling to be generated: DatasetID(name='true_color', wavelength=None, resolution=None, polarization=None, calibration=None, level=None, modifiers=None)
Traceback (most recent call last):
  File "c:\Users\graha\miniconda3\envs\pytroll\lib\site-packages\satpy\readers\__init__.py", line 301, in __getitem__
    return super(DatasetDict, self).__getitem__(item)
KeyError: 'true_color'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "T:\PythonScripts\WindowsScripts\FY-3D_VIS.py", line 160, in <module>
    scn.save_dataset(composite,imgdir+'/FY-3D-Orbit_'+str(orbmax)+'-'+
  File "c:\Users\graha\miniconda3\envs\pytroll\lib\site-packages\satpy\scene.py", line 1302, in save_dataset
    return writer.save_dataset(self[dataset_id],
  File "c:\Users\graha\miniconda3\envs\pytroll\lib\site-packages\satpy\scene.py", line 679, in __getitem__
    return self.datasets[key]
  File "c:\Users\graha\miniconda3\envs\pytroll\lib\site-packages\satpy\readers\__init__.py", line 303, in __getitem__
    key = self.get_key(item)
  File "c:\Users\graha\miniconda3\envs\pytroll\lib\site-packages\satpy\readers\__init__.py", line 290, in get_key
    return get_key(match_key, self.keys(), num_results=num_results,
  File "c:\Users\graha\miniconda3\envs\pytroll\lib\site-packages\satpy\readers\__init__.py", line 245, in get_key
    raise KeyError("No dataset matching '{}' found".format(str(key)))
KeyError: "No dataset matching 'DatasetID(name='true_color', wavelength=None, resolution=None, polarization=None, calibration=None, level=None, modifiers=None)' found"


It doesnt mean much to me but it may to you

Kind Regards

Graham


Re: Pytroll/SatPy for EUMETCast

Douglas Deans
 

Douglas
Just download the zipfile attached to message:
https://groups.io/g/MSG-1/message/29367
Maybe the attachment was to big for your e-mailer
Ernst
==================================================

Thanks Ernst. I found it on the Groups io page.
Some busy days ahead I expect for a programming novice but I appreciate the effort you have put into this.

Regards,
Douglas.


Re: Pytroll/SatPy for EUMETCast

Ernst Lobsiger
 

On Wed, May 6, 2020 at 09:15 AM, Douglas Deans wrote:
On 06/05/2020 16:52, Christian Peters via groups.io wrote:
Ernst,

really nice package and a lot of work!
Thanks for providing this, I hope someone will start to use Python and
Pytroll/Satpy.
It's a lot of fun...even if it's sometimes hard to fiddle with problems.

Welcome back to GNU/Linux!

Regards,

Christian
====================================================================

Christian when was Ernst's e-mail sent. I have not received it nor is it
in my ISP's spam folder.

Regards,
Douglas.
Douglas

Just download the zipfile attached to message:

https://groups.io/g/MSG-1/message/29367

Maybe the attachment was to big for your e-mailer

Ernst


Re: Pytroll/SatPy for EUMETCast

Douglas Deans
 

On 06/05/2020 16:52, Christian Peters via groups.io wrote:
Ernst,
really nice package and a lot of work!
Thanks for providing this, I hope someone will start to use Python and Pytroll/Satpy.
It's a lot of fun...even if it's sometimes hard to fiddle with problems.
Welcome back to GNU/Linux!
Regards,
Christian
====================================================================

Christian when was Ernst's e-mail sent. I have not received it nor is it in my ISP's spam folder.

Regards,
Douglas.


Re: Pytroll/SatPy for EUMETCast

Christian Peters
 

Ernst,

really nice package and a lot of work!
Thanks for providing this, I hope someone will start to use Python and Pytroll/Satpy.
It's a lot of fun...even if it's sometimes hard to fiddle with problems.

Welcome back to GNU/Linux! 

Regards,

Christian

Am 05.05.20 um 15:06 schrieb Ernst Lobsiger via groups.io:

Dear All

Here comes the Pytroll/Satpy EUMETCast Windows Starter Kit as promised. It should be "Childs Play" to get it working.
I have double checked all the scripts on my Windows 10 Pro 64Bit PC that got the files readonly from a SAMBA share Z:.

My definition of "Childs Play" (for Douglas):

 0) You know that you have Windows 10 64Bit
 1) You are not completley new to EUMETCast
 2) You read and follow the HOWTO inside (RTFM)
 3) You know where you find your satellite files

The xyRITDecompress.exe might be a temporary solution. There has been no reaction from EUMETSAT so far.
Maybe Hugo can provide a far better Windows solution that does not depend on any additional *.dll file.
This Starter Kit provides scripts for 14 satellites. You might not receive them all with your station.
Pytroll/Satpy will probably be one of the first packages to process MTG files. You better warm up now.

@Graham Woolf: As you cracked that and will crack that very fast you could also give Douglas a hand?
@Christian Peters: Now I'll go right back to GNU/Linux. MC is far easier than this Windows 10 Explorer!

Good Luck
Ernst

P.S Please also report SUCCESS in ths thread :-) ...


Re: Pytroll/SatPy for EUMETCast

Graham Woolf
 

Hi Ernst

Thank you so much for that

I will have a look tomorrow as I am busy this afternoon

Hopefully they will be pretty similar to the files I have working at the moment

I will try my best to help but I am just as much a python novice as most I would think

Kind Regards

Graham


Re: Pytroll/SatPy for EUMETCast

Ernst Lobsiger
 

Dear All

Here comes the Pytroll/Satpy EUMETCast Windows Starter Kit as promised. It should be "Childs Play" to get it working.
I have double checked all the scripts on my Windows 10 Pro 64Bit PC that got the files readonly from a SAMBA share Z:.

My definition of "Childs Play" (for Douglas):

 0) You know that you have Windows 10 64Bit
 1) You are not completley new to EUMETCast
 2) You read and follow the HOWTO inside (RTFM)
 3) You know where you find your satellite files

The xyRITDecompress.exe might be a temporary solution. There has been no reaction from EUMETSAT so far.
Maybe Hugo can provide a far better Windows solution that does not depend on any additional *.dll file.
This Starter Kit provides scripts for 14 satellites. You might not receive them all with your station.
Pytroll/Satpy will probably be one of the first packages to process MTG files. You better warm up now.

@Graham Woolf: As you cracked that and will crack that very fast you could also give Douglas a hand?
@Christian Peters: Now I'll go right back to GNU/Linux. MC is far easier than this Windows 10 Explorer!

Good Luck
Ernst

P.S Please also report SUCCESS in ths thread :-) ...


Re: Pytroll/SatPy for EUMETCast

Hugo
 

Hi Ernst and Rob,

I compiled my EUMETCastView and PublicDecompWT_2.06 under Windows with a version of Mingw-w64 a long time ago and I used that library ever since.
For the moment I don't have access to the Windows 7 computer where I used to compile my windows versions of EUMETCastView. (I am for the moment in a different place ...).
I'am using a Windows 10 computer now,  but I need to reinstall everything to recompile EUMETCastView and  PublicDecompWT_2.06 .  I know that I made some changes to the code of  PublicDecompWT_2.06 , just to be able to compile it.
As soon as I get back to that Windows 7 computer I will publish the complete code of EUMETCastView  (and   PublicDecompWT_2.06)  as a Qt project on Github.

Kind regards,

Hugo 


Re: Pytroll/SatPy for EUMETCast

Ernst Lobsiger
 

Rob,

you are talking about mingw. I tried different versions of mingw under GNU/Linux and Cygwin but compilation of xRITDecompress always failed.
With mingw I have no clue anyway. Hugo might be of help here. I finally compiled with the latest versions of Cygwin32 (on Windows 7 32Bit) and
Cygwin64 (on Windows 10 64Bit) with nothing but binutils, gcc-g++ compiler and make added to the defaults. There was no mingw whatsoever.

Regards,
Ernst


Re: Pytroll/SatPy for EUMETCast

R. Alblas
 

Ernst,

I did nothing specials. The makefiles for my software are exactly the same for Linux and Windows (except for a very few #define's). For the rest it's plane gcc/g++ etc.
But cygwin 32 is quite old (3.2.17, mingw 3.4.5); I don't know if something changed in the mean time.
Anyway, I didn't have a good look at the makefiles of xRITDecompress. On Linux it did compile without problems, but in Cygwin it failed here.

Regards,
Rob.

On 05/04/2020 02:11 PM, Ernst Lobsiger via groups.io wrote:
Rob,

not sure how you managed to do that. I just used the standard makefile(s) that come with xRITDecompress.
I only changed the default BITS=32 to 64 for Windows 10 64Bit. Maybe it's possible to statically link the executable.
But searching on the internet I found that even a simple hello world program would at least need cygwin1.dll.

Regards,
Ernst

P.S. I had the same behaviour with Cygwin32 with 3 slightly differently named *.dll (32 Bit version already trashed)


Re: Pytroll/SatPy for EUMETCast

Ernst Lobsiger
 

Rob,

not sure how you managed to do that. I just used the standard makefile(s) that come with xRITDecompress.
I only changed the default BITS=32 to 64 for Windows 10 64Bit. Maybe it's possible to statically link the executable.
But searching on the internet I found that even a simple hello world program would at least need cygwin1.dll.

Regards,
Ernst

P.S. I had the same behaviour with Cygwin32 with 3 slightly differently named *.dll (32 Bit version already trashed)


Re: Pytroll/SatPy for EUMETCast

R. Alblas
 

Hi, Ernst.

Just out of curiosity: I wondered why dll's are needed. I compile my software for Windows using cygwin 32, and the executable doesn't need any cygwin dll's. That includes the wavelet software part of xrit2pic.
Apparently "something" extra is needed by xRITDecompress.exe?

Regards,
Rob.


On 05/02/2020 11:58 PM, Ernst Lobsiger via groups.io wrote:
Douglas, Graham and All

I managed to compile xRITDecompress.exe on an old Windows-7 + Cygwin box as a 32Bit application.
It runs on Windows 10 PRO 64Bit with 3 Cygwin *.dll files at the side. David will certainly say that this is
a "botch". But it is a temporary solution (no reaction from EUMETSAT so far) that works. I have now a
dozen satellites (including Meteosat 8 and 11) that I can process with my Satpy scripts under Windows.

It will take me 2-3 more days to finish this Windows Pytroll/Satpy for EUMETCast distibution. Stay tuned.

Best Regards
Ernst


Re: Pytroll/SatPy for EUMETCast

Christian Peters
 

David,

you are right....I overlooked this! I seldom use the Africa part, see pic.

I tried sharpening with the HRV channel in SatPy but there is some problem with the colors if I use the built-in method.
Maybe some external sharpening with IM could improve this quite a bit?

Just some room to enhance...but not for me as I'm no Python programmer...! ;-)

Regard,

Christian



Am 03.05.20 um 20:23 schrieb David J Taylor via groups.io:

Graham,

Ernst is right, HRV has a higher resolution, you have to resample like:

area = 'eurol'
new_scene = global_scene.resample(area, radius_of_influence = 20000, cache_dir='/tmp')

You could even resample to fulldisk...

area = 'seviri_0deg'

but I think it makes no sense as the HRV channel corvers only the north of the earth..but just try it...

Regards,
Christian
=======================================

HRV covers both Europe and Africa.  You may the thinking of Rapid-Scan which is north only.  Take a look at this video with e.g. VLC Media Player:

 http://www.eumetsat.int/website/wcm/idc/idcplg?IdcService=GET_FILE&dDocName=AVI_050622_0000-2400_M8-CH12&RevisionSelectionMethod=LatestReleased&Rendition=Web

Cheers,
David


Re: Pytroll/SatPy for EUMETCast

David J Taylor GM8ARV 🏴󠁧󠁢󠁳󠁣󠁴󠁿 🇪🇺
 

It would be great if this was written up by the group - perhaps in the EUMETCast Wiki which allows multi-authoring (and is rather in need of updates!).

http://eumetcast.pbworks.com/w/page/8278549/FrontPage

Cheers,
David
--
SatSignal Software - Quality software for you
Web: https://www.satsignal.eu
Email: david-taylor@...
Twitter: @gm8arv


Re: Pytroll/SatPy for EUMETCast

Ernst Lobsiger
 

Christian and Graham

From ESA (Seviri instrument):
The 1 km sampling at SSP of the HRV channel is achieved by using 9 broadband detection elements. The other channels are sampled at 3 km SSP by using 3 narrow-band detection elements per channel.

This Seviri resolution is at SSP (0°/0°). In the UK it is certainly less. So the best images we can produce is probably 'HRV' only with 'eurol1'. You can adapt your own projection like 'westminster' for best results.

Regards,
Ernst


Re: Pytroll/SatPy for EUMETCast

David J Taylor GM8ARV 🏴󠁧󠁢󠁳󠁣󠁴󠁿 🇪🇺
 

From: Christian Peters via groups.io

ahhh.... IR_016, VIS008 and VIS006 ....!!!

last post...! :-D
=====================================

And you can enhance that by adding the detail which comes from differentiating the HRV, for at least the HRV coverage.

I do wish that wasn't called "Natural Colour" by EUMETSAT, because snow and ice are /not/ cyan. Sigh! Everybody uses it, though.

Cheers,
David
--
SatSignal Software - Quality software for you
Web: https://www.satsignal.eu
Email: david-taylor@...
Twitter: @gm8arv


Re: Pytroll/SatPy for EUMETCast

David J Taylor GM8ARV 🏴󠁧󠁢󠁳󠁣󠁴󠁿 🇪🇺
 

Graham,

Ernst is right, HRV has a higher resolution, you have to resample like:

area = 'eurol'
new_scene = global_scene.resample(area, radius_of_influence = 20000, cache_dir='/tmp')

You could even resample to fulldisk...

area = 'seviri_0deg'

but I think it makes no sense as the HRV channel corvers only the north of the earth..but just try it...

Regards,
Christian
=======================================

HRV covers both Europe and Africa. You may the thinking of Rapid-Scan which is north only. Take a look at this video with e.g. VLC Media Player:

http://www.eumetsat.int/website/wcm/idc/idcplg?IdcService=GET_FILE&dDocName=AVI_050622_0000-2400_M8-CH12&RevisionSelectionMethod=LatestReleased&Rendition=Web

Cheers,
David
--
SatSignal Software - Quality software for you
Web: https://www.satsignal.eu
Email: david-taylor@...
Twitter: @gm8arv


Re: Pytroll/SatPy for EUMETCast

Christian Peters
 

ahhh.... IR_016, VIS008 and VIS006 ....!!!

last post...! :-D

Am 03.05.20 um 19:51 schrieb Christian Peters:

..sorry,

you need IR_016, VIS008 and VIS008 for
'natural_enh'
.
... no HRV.... but nice at fulldisk...


Am 03.05.20 um 19:24 schrieb Christian Peters:
Graham and Ernst,

yes...there is no high resolution realistic color composite available.
We have to wait for the MTG series...! :-D

You could try the

'hrv_clouds' (needs HRV and IR_108)

or

'natural_enh' (needs HRV and
IR_016)


composite, but no true color at all.

Looks nice.

Have a look at
https://github.com/pytroll/satpy/blob/master/satpy/etc/composites/seviri.yaml
an try some composites for MSG...a lot of fun! :-)

Regards,

Christian



Am 03.05.20 um 19:09 schrieb Ernst Lobsiger via groups.io:
Graham,

you are making remarkable progress! This is my area 'westminster' from today with realistic colors!


Cheers,
Ernst




Re: Pytroll/SatPy for EUMETCast

Christian Peters
 

..sorry,

you need IR_016, VIS008 and VIS008 for
'natural_enh'
.
... no HRV.... but nice at fulldisk...


Am 03.05.20 um 19:24 schrieb Christian Peters:

Graham and Ernst,

yes...there is no high resolution realistic color composite available.
We have to wait for the MTG series...! :-D

You could try the

'hrv_clouds' (needs HRV and IR_108)

or

'natural_enh' (needs HRV and
IR_016)


composite, but no true color at all.

Looks nice.

Have a look at
https://github.com/pytroll/satpy/blob/master/satpy/etc/composites/seviri.yaml
an try some composites for MSG...a lot of fun! :-)

Regards,

Christian



Am 03.05.20 um 19:09 schrieb Ernst Lobsiger via groups.io:
Graham,

you are making remarkable progress! This is my area 'westminster' from today with realistic colors!


Cheers,
Ernst