Date   
Re: Pytroll/SatPy for EUMETCast

Ernst Lobsiger
 

Graham,

have a look into the polar scripts. The first scene is 'scn' loading the raw sat data ('swath').
Then we make different areas in special projection 'nscn' is a new scene produced by resampling.
We resample 'omerc_bb' and 'eurol' just as examples.

For geostationary sats the basic scene is the full earth scan data. For Europe resample 'eurol'.

Forget the /tmp to begin with. Use just:

area='eurol'
new_scene = global_scene.resample(area, radius_of_influence = 20000)

Then save the image from the new_scene


Good luck
Ernst

Re: Pytroll/SatPy for EUMETCast

Graham Woolf
 

Hi Christian

Im not sure of the correct syntax to get this to work
I have tried

area = 'eurol'
new_scene = global_scene.resample(area, radius_of_influence = 20000,  cache_dir=tmpdir)

global_scene = new_scene(filenames = files, reader = 'seviri_l1b_hrit')
global_scene.load([composite])

but this just fails

Regards

Graham

Re: Pytroll/SatPy for EUMETCast

Graham Woolf
 

Hi Christian

Thanks I will.

I you have probably gathered I am a complete novice at python but am slowly feeling my way around with much help from Ernst

Kind Regards

Graham

Re: Pytroll/SatPy for EUMETCast

Christian Peters
 

Graham,

Ernst is right, HRV has a higher resolution, you have to resample like:

area = 'eurol'
new_scene = global_scene.resample(area, radius_of_influence = 20000, cache_dir='/tmp')

You could even resample to fulldisk...

area = 'seviri_0deg'

but I think it makes no sense as the HRV channel corvers only the north of the earth..but just try it...


Regards,

Christian

Am 03.05.20 um 17:22 schrieb Graham Woolf:

Hi Ernst

I was having a read of the links you sent and thought i would try this

I found this in the seviri.yami

  realistic_colors:
    compositor: !!python/name:satpy.composites.RealisticColors
    standard_name: realistic_colors
    prerequisites:
      - name: VIS006
        modifiers: [sunz_corrected]
      - name: VIS008
        modifiers: [sunz_corrected]
      - name: HRV
        modifiers: [sunz_corrected]

so i modified my script as below

# Composite 'realistic colors' use the 3 channels in needed_files
composite = 'realistic_colors'
needed_files = ['VIS006', 'VIS008', 'HRV']

I then ran the script and got the error

Traceback (most recent call last):
  File "c:\Users\graha\miniconda3\envs\pytroll\lib\site-packages\satpy\readers\__init__.py", line 301, in __getitem__
    return super(DatasetDict, self).__getitem__(item)
KeyError: 'realistic_colors'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "T:\PythonScripts\WindowsScripts\msg4realistic.py", line 88, in <module>
    global_scene.save_dataset(composite,imgdir + "/" + "MSG4.png",
  File "c:\Users\graha\miniconda3\envs\pytroll\lib\site-packages\satpy\scene.py", line 1302, in save_dataset
    return writer.save_dataset(self[dataset_id],
  File "c:\Users\graha\miniconda3\envs\pytroll\lib\site-packages\satpy\scene.py", line 679, in __getitem__
    return self.datasets[key]
  File "c:\Users\graha\miniconda3\envs\pytroll\lib\site-packages\satpy\readers\__init__.py", line 303, in __getitem__
    key = self.get_key(item)
  File "c:\Users\graha\miniconda3\envs\pytroll\lib\site-packages\satpy\readers\__init__.py", line 290, in get_key
    return get_key(match_key, self.keys(), num_results=num_results,
  File "c:\Users\graha\miniconda3\envs\pytroll\lib\site-packages\satpy\readers\__init__.py", line 245, in get_key
    raise KeyError("No dataset matching '{}' found".format(str(key)))
KeyError: "No dataset matching 'DatasetID(name='realistic_colors', wavelength=None, resolution=None, polarization=None, calibration=None, level=None, modifiers=None)' found"


So I'm not clear what the relationship is between the composite and the reader

If I change the composite to I have to change the reader

Regards

Graham

Re: Pytroll/SatPy for EUMETCast

Ernst Lobsiger
 

Graham,

the problem certainly is that HRV is a different resolution and does not cover the whole disk.
This composite probably works only for projection "eurol" and the like (may need resampling too).

I'll look into that.

Ernst

Re: Pytroll/SatPy for EUMETCast

Graham Woolf
 

Hi Ernst

I was having a read of the links you sent and thought i would try this

I found this in the seviri.yami

  realistic_colors:
    compositor: !!python/name:satpy.composites.RealisticColors
    standard_name: realistic_colors
    prerequisites:
      - name: VIS006
        modifiers: [sunz_corrected]
      - name: VIS008
        modifiers: [sunz_corrected]
      - name: HRV
        modifiers: [sunz_corrected]

so i modified my script as below

# Composite 'realistic colors' use the 3 channels in needed_files
composite = 'realistic_colors'
needed_files = ['VIS006', 'VIS008', 'HRV']

I then ran the script and got the error

Traceback (most recent call last):
  File "c:\Users\graha\miniconda3\envs\pytroll\lib\site-packages\satpy\readers\__init__.py", line 301, in __getitem__
    return super(DatasetDict, self).__getitem__(item)
KeyError: 'realistic_colors'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "T:\PythonScripts\WindowsScripts\msg4realistic.py", line 88, in <module>
    global_scene.save_dataset(composite,imgdir + "/" + "MSG4.png",
  File "c:\Users\graha\miniconda3\envs\pytroll\lib\site-packages\satpy\scene.py", line 1302, in save_dataset
    return writer.save_dataset(self[dataset_id],
  File "c:\Users\graha\miniconda3\envs\pytroll\lib\site-packages\satpy\scene.py", line 679, in __getitem__
    return self.datasets[key]
  File "c:\Users\graha\miniconda3\envs\pytroll\lib\site-packages\satpy\readers\__init__.py", line 303, in __getitem__
    key = self.get_key(item)
  File "c:\Users\graha\miniconda3\envs\pytroll\lib\site-packages\satpy\readers\__init__.py", line 290, in get_key
    return get_key(match_key, self.keys(), num_results=num_results,
  File "c:\Users\graha\miniconda3\envs\pytroll\lib\site-packages\satpy\readers\__init__.py", line 245, in get_key
    raise KeyError("No dataset matching '{}' found".format(str(key)))
KeyError: "No dataset matching 'DatasetID(name='realistic_colors', wavelength=None, resolution=None, polarization=None, calibration=None, level=None, modifiers=None)' found"


So I'm not clear what the relationship is between the composite and the reader

If I change the composite to I have to change the reader

Regards

Graham

Re: Pytroll/SatPy for EUMETCast

Ernst Lobsiger
 

Graham,

nice to hear that all is working now. So we have a Win10 SUCCESS!


"Why do they do this?" As I told you in my first Pytroll/Satpy HOWTO:
<cite>
There is an important note (in the ever updated SatPy documentation):
*****************************************************************************************
* "SatPy's interfaces are not guaranteed stable and may change until *
* version 1.0 when backwards compatibility will be a main focus."         *
*****************************************************************************************
</cite>

SatPy is now Version 0.21 still in full developpment. We have to live
with changes then and when. We must have a minimal understanding
of Python and know where to look either in documentation or in code.

On the other hand we can define how we want the results to look like!

Cheers,
Ernst

Re: Pytroll/SatPy for EUMETCast

Graham Woolf
 

Hi Ernst

Thanks - I did spot that one but managed to work it out

I ll have a look at those links - thanks

Regards

Graham

Re: Pytroll/SatPy for EUMETCast

Graham Woolf
 

Hi Ernst

Why do they do that ? do they tell anybody

Anyway all working now - they did the same with the GOMS reader but I managed to work it out

Many thanks again

Kind Regards

Graham

Re: Pytroll/SatPy for EUMETCast

Ernst Lobsiger
 

Graham,

you may have noticed there is one more problem changing from GNU/Linux to Windows:
GNU/Linux uses command -switch parameters Windows command parameters /switch

Before you decompress the files you must clear the respective tmp directory. We have
to reverse the order from "del cd" to "cd del". So for Meteosat  Python scripts it's now:

# Dir must exist
os.chdir(tmpdir)
# Clean out the tmp directory first
os.system("del H-000*")

Where toi go from here:

What readers are available? Have a look (but don't screw up) here:
C:\users\username\miniconda3\envs\pytroll\Lib\site-packages\satpy\readers

What composites need what instrument channels? Find answers here:
C:\users\username\miniconda3\envs\pytroll\Lib\site-packages\satpy\etc\composites

Read the docs ...


Best Regards,
Ernst

Re: Pytroll/SatPy for EUMETCast

Ernst Lobsiger
 

Graham,
yes they have renamed some readers and composites. Try:
...
# For composite 'natural_color' we use three channels in needed_files
composite = 'natural_color'
needed_files = ['VIS006', 'VIS008', 'IR_016']
...

Regards,
Ernst

Re: Pytroll/SatPy for EUMETCast

Graham Woolf
 

Hi Ernst

OK done that now get

Traceback (most recent call last):
  File "T:\PythonScripts\WindowsScripts\MSG4.py", line 84, in <module>
    global_scene.load([composite])
  File "c:\Users\graha\miniconda3\envs\pytroll\lib\site-packages\satpy\scene.py", line 969, in load
    raise KeyError("Unknown datasets: {}".format(unknown_str))
KeyError: 'Unknown datasets: natural_sun'

Kind Regards

Graham

Re: Pytroll/SatPy for EUMETCast

Ernst Lobsiger
 

Graham, Try:

global_scene = Scene(filenames = files, reader = 'seviri_l1b_hrit')

Cheers
Ernst

Re: Pytroll/SatPy for EUMETCast

Graham Woolf
 

Hi Ernst

I Have tried to create an MSG4 image but get the following error

(pytroll) C:\WINDOWS\system32>python T:\PythonScripts\WindowsScripts\MSG4.py 202005031030
Decompressed file: H-000-MSG4__-MSG4________-VIS006___-000006___-202005031030-__
Decompressed file: H-000-MSG4__-MSG4________-VIS006___-000004___-202005031030-__
Decompressed file: H-000-MSG4__-MSG4________-VIS006___-000005___-202005031030-__
Decompressed file: H-000-MSG4__-MSG4________-VIS006___-000001___-202005031030-__
Decompressed file: H-000-MSG4__-MSG4________-VIS006___-000002___-202005031030-__
Decompressed file: H-000-MSG4__-MSG4________-VIS006___-000008___-202005031030-__
Decompressed file: H-000-MSG4__-MSG4________-VIS006___-000007___-202005031030-__
Decompressed file: H-000-MSG4__-MSG4________-VIS006___-000003___-202005031030-__
Decompressed file: H-000-MSG4__-MSG4________-VIS008___-000006___-202005031030-__
Decompressed file: H-000-MSG4__-MSG4________-VIS008___-000003___-202005031030-__
Decompressed file: H-000-MSG4__-MSG4________-VIS008___-000008___-202005031030-__
Decompressed file: H-000-MSG4__-MSG4________-VIS008___-000001___-202005031030-__
Decompressed file: H-000-MSG4__-MSG4________-VIS008___-000005___-202005031030-__
Decompressed file: H-000-MSG4__-MSG4________-VIS008___-000002___-202005031030-__
Decompressed file: H-000-MSG4__-MSG4________-VIS008___-000007___-202005031030-__
Decompressed file: H-000-MSG4__-MSG4________-VIS008___-000004___-202005031030-__
Decompressed file: H-000-MSG4__-MSG4________-IR_016___-000005___-202005031030-__
Decompressed file: H-000-MSG4__-MSG4________-IR_016___-000007___-202005031030-__
Decompressed file: H-000-MSG4__-MSG4________-IR_016___-000008___-202005031030-__
Decompressed file: H-000-MSG4__-MSG4________-IR_016___-000003___-202005031030-__
Decompressed file: H-000-MSG4__-MSG4________-IR_016___-000002___-202005031030-__
Decompressed file: H-000-MSG4__-MSG4________-IR_016___-000006___-202005031030-__
Decompressed file: H-000-MSG4__-MSG4________-IR_016___-000004___-202005031030-__
Decompressed file: H-000-MSG4__-MSG4________-IR_016___-000001___-202005031030-__
Traceback (most recent call last):
  File "T:\PythonScripts\WindowsScripts\MSG4.py", line 80, in <module>
    global_scene = Scene(filenames = files, reader = 'hrit_msg')
  File "c:\Users\graha\miniconda3\envs\pytroll\lib\site-packages\satpy\scene.py", line 149, in __init__
    self.readers = self.create_reader_instances(filenames=filenames,
  File "c:\Users\graha\miniconda3\envs\pytroll\lib\site-packages\satpy\scene.py", line 193, in create_reader_instances
    return load_readers(filenames=filenames,
  File "c:\Users\graha\miniconda3\envs\pytroll\lib\site-packages\satpy\readers\__init__.py", line 703, in load_readers
    for idx, reader_configs in enumerate(configs_for_reader(reader, ppp_config_dir)):
  File "c:\Users\graha\miniconda3\envs\pytroll\lib\site-packages\satpy\readers\__init__.py", line 549, in configs_for_reader
    raise ValueError("No reader(s) named: {}".format(reader))
ValueError: No reader(s) named: ['hrit_msg']

(pytroll) C:\WINDOWS\system32>call c:\users\graha\miniconda3\Scripts\deactivate pytroll
'c:\users\graha\miniconda3\Scripts\deactivate' is not recognized as an internal or external command,
operable program or batch file.

It looks as if it cant find this reader

No reader(s) named: ['hrit_msg']

Any Ideas

Kind Regards

Graham






Re: Pytroll/SatPy for EUMETCast

Ernst Lobsiger
 

Graham,

>Cite>
I have all of mine working now apart from the ones requiring xRITDecompress
</Cite>

You certainly can't wait. Here is what you need for Meteosat XX and GOMS2:

Good Luck
Ernst

Re: Pytroll/SatPy for EUMETCast

Ernst Lobsiger
 

Graham and All,

DON'T install Cygwin64. It's free but it's UNIX for Windows. I will distribute xRITDecompress.exe and the 3 *.dll needed.
This can be done as the xRITDecompress c++ source code is now open-source and all Cygwin64 is under the GPL3.

YES, the scripts can certainly be scheduled via batch files that activate environment pytroll first. I'am not familiar
with how this is done under Windows 10 64Bit. Under GNU/Linux there is service cron as allmighty scheduler ...

Regards,
Ernst

Re: Pytroll/SatPy for EUMETCast

Graham Woolf
 

Hi Ernst

 

Not sure I understand all this – where do you get the source code from to compile it . I have never used Cygwin so dont know what it does

 

Glad you are getting on OK with the scripts – I look forward to seeing them. I have all of mine working now apart from the ones requiring xRITDecompress

How to you run yours - do you run them manually every day or do you use Task Scheduler

Stay Safe

 

Kind Regards Graham

Re: Pytroll/SatPy for EUMETCast

Ernst Lobsiger
 

Dear All

Different from what has been said and tried by others in this thread it is easy to compile xRITDecompress as a genuine 64Bit executable under Cygwin64.
I just tested Cygwin64 latest Version 3.1.4. I made a minimalistic install with packages binutils, gcc-g++ and make. In the main makefile I canged BITS=64.
I typed make and that's it! I copied xRITDecompress.exe to a folder on the desktop. If you click on it Windows tells you what *.dll are not found or missing.
From C:\Cygwin64\bin I had to copy the files cyggcc_s-seh-1.dll, cygstdc++-6.dll and cygwin1.dll to the folder on the Desktop with xRITDecompress.exe.

By keeping these 3 *.dll aside xRITDecompress.exe Windows 10 64Bit does find all it needs without install of Cygwin64 or spoiling the registry needed :-).

Best Regards
Ernst

Re: Pytroll/SatPy for EUMETCast

Ernst Lobsiger
 

Douglas, Graham and All

I managed to compile xRITDecompress.exe on an old Windows-7 + Cygwin box as a 32Bit application.
It runs on Windows 10 PRO 64Bit with 3 Cygwin *.dll files at the side. David will certainly say that this is
a "botch". But it is a temporary solution (no reaction from EUMETSAT so far) that works. I have now a
dozen satellites (including Meteosat 8 and 11) that I can process with my Satpy scripts under Windows.

It will take me 2-3 more days to finish this Windows Pytroll/Satpy for EUMETCast distibution. Stay tuned.

Best Regards
Ernst

Re: EUMETCAST Loss of High Volume Services

David J Taylor
 

Hello everyone,
[]
Typically, we are getting the hvs-1/EPS1, EPS2, EPS5, EPS10 data during the hour
between 06:00 and 07:00 but almost nothing outside that period.
[]
Regards,
Peter Chiu
========================================

With everything else discussed, it suggests the disk is misaligned and only near enough to correct pointing during that period. plotting Link Margin versus time of day would be interesting:

https://www.satsignal.eu/wxsat/dvb-s2/sr1-mrtg.html

Cheers,
David
--
SatSignal Software - Quality software for you
Web: https://www.satsignal.eu
Email: david-taylor@...
Twitter: @gm8arv