Date   

Re: Pytroll/SatPy for EUMETCast

Ernst Lobsiger
 

Graham,

you get again ahead of yourself. Area projections is not exactly the easiest point in Pytroll/Satpy.

On Sun, May 3, 2020 at 10:06 AM, Graham Woolf wrote:
Well I have produced an image but it doesnt seem very high resolution like the HRV should be
1.) HRV is only one channel, the other channels you use for 'realitic_colors' are lower resolution.
2.) The size of the image in pixels depends on the entry in area.yaml  (eurol is rather for 3km).

Try e.g. 'euron0250' instead of 'eurol' (you better go for a cup of tea while resampling ...:-))

Cheers,
Ernst


Re: Pytroll/SatPy for EUMETCast

Graham Woolf
 

Hi Ernst

I'll let you into a little secret

I used to be a software engineer in a past life but havent done any programming for about 20 years

I worked for National Air Traffic Services writing ADA for simulators

So I feel I am struggling a bit

Kind Regards

Graham


Re: Pytroll/SatPy for EUMETCast

Graham Woolf
 

Hi Ernst

Lovely image

Can you show me the script that produced that ?

Many thanks

Graham


Re: Pytroll/SatPy for EUMETCast

Christian Peters
 

Graham and Ernst,

yes...there is no high resolution realistic color composite available.
We have to wait for the MTG series...! :-D

You could try the

'hrv_clouds' (needs HRV and IR_108)

or

'natural_enh' (needs HRV and
IR_016)


composite, but no true color at all.

Looks nice.

Have a look at
https://github.com/pytroll/satpy/blob/master/satpy/etc/composites/seviri.yaml
an try some composites for MSG...a lot of fun! :-)

Regards,

Christian



Am 03.05.20 um 19:09 schrieb Ernst Lobsiger via groups.io:

Graham,

you are making remarkable progress! This is my area 'westminster' from today with realistic colors!


Cheers,
Ernst


Re: Pytroll/SatPy for EUMETCast

Ernst Lobsiger
 

Graham,

you are making remarkable progress! This is my area 'westminster' from today with realistic colors!


Cheers,
Ernst


Re: Pytroll/SatPy for EUMETCast

Graham Woolf
 

Hi Both

Well I have produced an image but it doesnt seem very high resolution like the HRV should be

Regards

Graham


Re: Pytroll/SatPy for EUMETCast

Christian Peters
 

Graham,

I'm even a novice sing Python.

Do like Ernst say...but you first have load the scene, than resample (change the order).

composite = 'realistic_colors'
area='eurol'

global_scene = Scene(filenames = files, reader = 'seviri_l1b_hrit')
global_scene.load([composite])
new_scene = global_scene.resample(area, radius_of_influence = 20000)
new_scene.save_dataset(.....


Regards,

Christian


Am 03.05.20 um 18:23 schrieb Graham Woolf:

Hi Christian

Im not sure of the correct syntax to get this to work
I have tried

area = 'eurol'
new_scene = global_scene.resample(area, radius_of_influence = 20000,  cache_dir=tmpdir)

global_scene = new_scene(filenames = files, reader = 'seviri_l1b_hrit')
global_scene.load([composite])

but this just fails

Regards

Graham


Re: Pytroll/SatPy for EUMETCast

Ernst Lobsiger
 

Graham,

have a look into the polar scripts. The first scene is 'scn' loading the raw sat data ('swath').
Then we make different areas in special projection 'nscn' is a new scene produced by resampling.
We resample 'omerc_bb' and 'eurol' just as examples.

For geostationary sats the basic scene is the full earth scan data. For Europe resample 'eurol'.

Forget the /tmp to begin with. Use just:

area='eurol'
new_scene = global_scene.resample(area, radius_of_influence = 20000)

Then save the image from the new_scene


Good luck
Ernst


Re: Pytroll/SatPy for EUMETCast

Graham Woolf
 

Hi Christian

Im not sure of the correct syntax to get this to work
I have tried

area = 'eurol'
new_scene = global_scene.resample(area, radius_of_influence = 20000,  cache_dir=tmpdir)

global_scene = new_scene(filenames = files, reader = 'seviri_l1b_hrit')
global_scene.load([composite])

but this just fails

Regards

Graham


Re: Pytroll/SatPy for EUMETCast

Graham Woolf
 

Hi Christian

Thanks I will.

I you have probably gathered I am a complete novice at python but am slowly feeling my way around with much help from Ernst

Kind Regards

Graham


Re: Pytroll/SatPy for EUMETCast

Christian Peters
 

Graham,

Ernst is right, HRV has a higher resolution, you have to resample like:

area = 'eurol'
new_scene = global_scene.resample(area, radius_of_influence = 20000, cache_dir='/tmp')

You could even resample to fulldisk...

area = 'seviri_0deg'

but I think it makes no sense as the HRV channel corvers only the north of the earth..but just try it...


Regards,

Christian

Am 03.05.20 um 17:22 schrieb Graham Woolf:

Hi Ernst

I was having a read of the links you sent and thought i would try this

I found this in the seviri.yami

  realistic_colors:
    compositor: !!python/name:satpy.composites.RealisticColors
    standard_name: realistic_colors
    prerequisites:
      - name: VIS006
        modifiers: [sunz_corrected]
      - name: VIS008
        modifiers: [sunz_corrected]
      - name: HRV
        modifiers: [sunz_corrected]

so i modified my script as below

# Composite 'realistic colors' use the 3 channels in needed_files
composite = 'realistic_colors'
needed_files = ['VIS006', 'VIS008', 'HRV']

I then ran the script and got the error

Traceback (most recent call last):
  File "c:\Users\graha\miniconda3\envs\pytroll\lib\site-packages\satpy\readers\__init__.py", line 301, in __getitem__
    return super(DatasetDict, self).__getitem__(item)
KeyError: 'realistic_colors'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "T:\PythonScripts\WindowsScripts\msg4realistic.py", line 88, in <module>
    global_scene.save_dataset(composite,imgdir + "/" + "MSG4.png",
  File "c:\Users\graha\miniconda3\envs\pytroll\lib\site-packages\satpy\scene.py", line 1302, in save_dataset
    return writer.save_dataset(self[dataset_id],
  File "c:\Users\graha\miniconda3\envs\pytroll\lib\site-packages\satpy\scene.py", line 679, in __getitem__
    return self.datasets[key]
  File "c:\Users\graha\miniconda3\envs\pytroll\lib\site-packages\satpy\readers\__init__.py", line 303, in __getitem__
    key = self.get_key(item)
  File "c:\Users\graha\miniconda3\envs\pytroll\lib\site-packages\satpy\readers\__init__.py", line 290, in get_key
    return get_key(match_key, self.keys(), num_results=num_results,
  File "c:\Users\graha\miniconda3\envs\pytroll\lib\site-packages\satpy\readers\__init__.py", line 245, in get_key
    raise KeyError("No dataset matching '{}' found".format(str(key)))
KeyError: "No dataset matching 'DatasetID(name='realistic_colors', wavelength=None, resolution=None, polarization=None, calibration=None, level=None, modifiers=None)' found"


So I'm not clear what the relationship is between the composite and the reader

If I change the composite to I have to change the reader

Regards

Graham


Re: Pytroll/SatPy for EUMETCast

Ernst Lobsiger
 

Graham,

the problem certainly is that HRV is a different resolution and does not cover the whole disk.
This composite probably works only for projection "eurol" and the like (may need resampling too).

I'll look into that.

Ernst


Re: Pytroll/SatPy for EUMETCast

Graham Woolf
 

Hi Ernst

I was having a read of the links you sent and thought i would try this

I found this in the seviri.yami

  realistic_colors:
    compositor: !!python/name:satpy.composites.RealisticColors
    standard_name: realistic_colors
    prerequisites:
      - name: VIS006
        modifiers: [sunz_corrected]
      - name: VIS008
        modifiers: [sunz_corrected]
      - name: HRV
        modifiers: [sunz_corrected]

so i modified my script as below

# Composite 'realistic colors' use the 3 channels in needed_files
composite = 'realistic_colors'
needed_files = ['VIS006', 'VIS008', 'HRV']

I then ran the script and got the error

Traceback (most recent call last):
  File "c:\Users\graha\miniconda3\envs\pytroll\lib\site-packages\satpy\readers\__init__.py", line 301, in __getitem__
    return super(DatasetDict, self).__getitem__(item)
KeyError: 'realistic_colors'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "T:\PythonScripts\WindowsScripts\msg4realistic.py", line 88, in <module>
    global_scene.save_dataset(composite,imgdir + "/" + "MSG4.png",
  File "c:\Users\graha\miniconda3\envs\pytroll\lib\site-packages\satpy\scene.py", line 1302, in save_dataset
    return writer.save_dataset(self[dataset_id],
  File "c:\Users\graha\miniconda3\envs\pytroll\lib\site-packages\satpy\scene.py", line 679, in __getitem__
    return self.datasets[key]
  File "c:\Users\graha\miniconda3\envs\pytroll\lib\site-packages\satpy\readers\__init__.py", line 303, in __getitem__
    key = self.get_key(item)
  File "c:\Users\graha\miniconda3\envs\pytroll\lib\site-packages\satpy\readers\__init__.py", line 290, in get_key
    return get_key(match_key, self.keys(), num_results=num_results,
  File "c:\Users\graha\miniconda3\envs\pytroll\lib\site-packages\satpy\readers\__init__.py", line 245, in get_key
    raise KeyError("No dataset matching '{}' found".format(str(key)))
KeyError: "No dataset matching 'DatasetID(name='realistic_colors', wavelength=None, resolution=None, polarization=None, calibration=None, level=None, modifiers=None)' found"


So I'm not clear what the relationship is between the composite and the reader

If I change the composite to I have to change the reader

Regards

Graham


Re: Pytroll/SatPy for EUMETCast

Ernst Lobsiger
 

Graham,

nice to hear that all is working now. So we have a Win10 SUCCESS!


"Why do they do this?" As I told you in my first Pytroll/Satpy HOWTO:
<cite>
There is an important note (in the ever updated SatPy documentation):
*****************************************************************************************
* "SatPy's interfaces are not guaranteed stable and may change until *
* version 1.0 when backwards compatibility will be a main focus."         *
*****************************************************************************************
</cite>

SatPy is now Version 0.21 still in full developpment. We have to live
with changes then and when. We must have a minimal understanding
of Python and know where to look either in documentation or in code.

On the other hand we can define how we want the results to look like!

Cheers,
Ernst


Re: Pytroll/SatPy for EUMETCast

Graham Woolf
 

Hi Ernst

Thanks - I did spot that one but managed to work it out

I ll have a look at those links - thanks

Regards

Graham


Re: Pytroll/SatPy for EUMETCast

Graham Woolf
 

Hi Ernst

Why do they do that ? do they tell anybody

Anyway all working now - they did the same with the GOMS reader but I managed to work it out

Many thanks again

Kind Regards

Graham


Re: Pytroll/SatPy for EUMETCast

Ernst Lobsiger
 

Graham,

you may have noticed there is one more problem changing from GNU/Linux to Windows:
GNU/Linux uses command -switch parameters Windows command parameters /switch

Before you decompress the files you must clear the respective tmp directory. We have
to reverse the order from "del cd" to "cd del". So for Meteosat  Python scripts it's now:

# Dir must exist
os.chdir(tmpdir)
# Clean out the tmp directory first
os.system("del H-000*")

Where toi go from here:

What readers are available? Have a look (but don't screw up) here:
C:\users\username\miniconda3\envs\pytroll\Lib\site-packages\satpy\readers

What composites need what instrument channels? Find answers here:
C:\users\username\miniconda3\envs\pytroll\Lib\site-packages\satpy\etc\composites

Read the docs ...


Best Regards,
Ernst


Re: Pytroll/SatPy for EUMETCast

Ernst Lobsiger
 

Graham,
yes they have renamed some readers and composites. Try:
...
# For composite 'natural_color' we use three channels in needed_files
composite = 'natural_color'
needed_files = ['VIS006', 'VIS008', 'IR_016']
...

Regards,
Ernst


Re: Pytroll/SatPy for EUMETCast

Graham Woolf
 

Hi Ernst

OK done that now get

Traceback (most recent call last):
  File "T:\PythonScripts\WindowsScripts\MSG4.py", line 84, in <module>
    global_scene.load([composite])
  File "c:\Users\graha\miniconda3\envs\pytroll\lib\site-packages\satpy\scene.py", line 969, in load
    raise KeyError("Unknown datasets: {}".format(unknown_str))
KeyError: 'Unknown datasets: natural_sun'

Kind Regards

Graham


Re: Pytroll/SatPy for EUMETCast

Ernst Lobsiger
 

Graham, Try:

global_scene = Scene(filenames = files, reader = 'seviri_l1b_hrit')

Cheers
Ernst