Date   

Re: Pytroll/SatPy for EUMETCast

Christian Peters
 

Graham

cloudtop:
compositor: !!python/name:satpy.composites.GenericCompositor
prerequisites:
- 3.9
- 10.8
- 12.0
standard_name: cloudtop

see table from Ernst script:

# Channel | VIS006 | HRV | VIS008 | IR_016 | IR_039 | WV_062 |
# WaveLen | 0.635 | 0.7 | 0.81 | 1.64 | 3.90 | 6.25 |
# --------+--------+--------+--------+--------+--------+--------+
# Channel | WV_073 | IR_087 | IR_097 | IR_108 | IR_120 | IR_134 |
# WaveLen | 7.35 | 8.70 | 9.66 | 10.80 | 12.00 | 13.40 |

I woul recommend to load all! chnalles in a demo script and the you can play around with different composites and not have to take care toload the channles neede.

So for cloudtop you need:

IR_016, IR_108 and IR_120

Regards,

Christian

Am 07.05.20 um 18:10 schrieb Graham Woolf:

Hi Christian
How do I process the cloud top composites
There is no indication in visir.yaml of the channesl needed for processing
Presumably I need other files do I ?
Kind Regards
Graham


Re: Pytroll/SatPy for EUMETCast

Ernst Lobsiger
 

Graham,

On Thu, May 7, 2020 at 09:10 AM, Graham Woolf wrote:
There is no indication in visir.yaml of the channesl needed for processing
it's all in the seviri.yaml file I attached for you a few posts back. You *have to* RTFM.

 cloudtop:
    compositor: !!python/name:satpy.composites.GenericCompositor
    prerequisites:
    - name: IR_039
      modifiers: [co2_corrected]
    - IR_108
    - IR_120
    standard_name: cloudtop

  cloudtop_daytime:
    compositor: !!python/name:satpy.composites.GenericCompositor
    prerequisites:
    - name: IR_039
      modifiers: [nir_emissive]
    - IR_108
    - IR_120
    standard_name: cloudtop
and much more that you can do with:

#************************************************************************
#
# European geostationary satellite Meteosat-11 (MSG4) at 0.0 degrees East
#------------------------------------------------------------------------
# EUMETSAT is EUMETCast disseminating MSG4 data in EUMETSAT_Data_Channel_2
# Meteosat-11 (MSG4), at 0.0 noon is exactly 12:00 UTC, instrument "seviri"
#
# Channel | VIS006 |   HRV  | VIS008 | IR_016 | IR_039 | WV_062 |
# WaveLen |  0.635 |   0.7  |   0.81 |   1.64 |   3.90 |   6.25 |
# --------+--------+--------+--------+--------+--------+--------+
# Channel | WV_073 | IR_087 | IR_097 | IR_108 | IR_120 | IR_134 |
# WaveLen |   7.35 |   8.70 |   9.66 |  10.80 |  12.00 |  13.40 |
#
# CH-3123 Belp, 2020/05/05       License GPL3          (c) Ernst Lobsiger
#
#************************************************************************

Cheers,
Ernst


Re: Pytroll/SatPy for EUMETCast

Graham Woolf
 

Hi Christian

How do I process the cloud top composites

There is no indication in visir.yaml of the channesl needed for processing

Presumably I need other files do I ?

Kind Regards

Graham


Re: TP 1 data rate.

David J Taylor GM8ARV 🏴󠁧󠁢󠁳󠁣󠁴󠁿 🇪🇺
 

Hello All,

Does anyone know the current total and maximum possible data rate, that TP 1 can disseminate?

I saw a couple of bursts of + 74 Mbps yesterday around 12:19 UTC.

Also, my instances of 2-3 second 'bursts' of negative Mbps with the SR1 dials seems to be increasing.
Or perhaps I'm just spending too much time at the computer?

Regards,
John.
======================================

John,

Expand the EUMETCast Europe section here:

https://www.eumetsat.int/website/home/Data/DataDelivery/EUMETCast/ReceptionStationSetup/index.html

77 Mbps at the IP level. Same for TP2.

Cheers,
David
--
SatSignal Software - Quality software for you
Web: https://www.satsignal.eu
Email: david-taylor@...
Twitter: @gm8arv


Re: Pytroll/SatPy for EUMETCast

Ernst Lobsiger
 

Christian,

On Thu, May 7, 2020 at 08:35 AM, Christian Peters wrote:
If you process MSG, Satpy looks in seviri.yaml and in visir.yaml.
yes I know. All that Graham has to do is provide a list of needed_files=[...] to decompress
with the wavelengths in visir and the channel naming of the SEVIRI in the pyscript header.

Going for work in my garden now. No more corona(at) home schooling for some time.

Cheers,
Ernst


Re: Pytroll/SatPy for EUMETCast

Graham Woolf
 

Hi Christian

Thanks, found it now and have an image

Many thanks

Graham


Re: Pytroll/SatPy for EUMETCast

Christian Peters
 

Ernst and Graham,

you don't have to.
If you process MSG, Satpy looks in seviri.yaml and in visir.yaml.
So you can choose composites in visir.yaml too ...if the channels match.

Regards,

Christian

Am 07.05.20 um 17:24 schrieb Ernst Lobsiger via groups.io:

On Thu, May 7, 2020 at 07:56 AM, Graham Woolf wrote:
I have had a look and I cant see an airmass composite for Meteosat anywhere which is strange
Graham,

Maybe you can use the definition of the more generic VISIR "instruments". SEVIRI has the right stuff.

Regards,
Ernst


Re: Pytroll/SatPy for EUMETCast

Ernst Lobsiger
 

On Thu, May 7, 2020 at 07:56 AM, Graham Woolf wrote:
I have had a look and I cant see an airmass composite for Meteosat anywhere which is strange
Graham,

Maybe you can use the definition of the more generic VISIR "instruments". SEVIRI has the right stuff.

Regards,
Ernst


Re: Pytroll/SatPy for EUMETCast

Ernst Lobsiger
 

Graham,

I downloaded your files and I get the following:

With POI "Swizerland" I get the image attached, With POI "Isle od Man" I get the same problem as you.
Looking into the list of bestfiles I see only 2 Files with 1000m data but 3 GEO1k location files. Obviously
Pytroll/Satpy cannot handle that and obviously you should invest in better HVS-1 reception. Try my
TCLogSummary.cmd  (as member of Hugo's club I can't find it right now). David has used it for years.

One more idea: If you start the scripts with a scheduler beware if your system doesn't run on UTC.
David also has a *.exe for that problem but it doesn't run on Windows 10 64Bit without installation
of additional system stuff. That's why I could not (or did not) pack it with my PyTroll/Satpy starter kit.

https://www.satsignal.eu/software/disk.html#GetUtcDateTimeString

Cheers,
Ernst


Re: Pytroll/SatPy for EUMETCast

Graham Woolf
 

Hi Ernst

Should have thought of that

I have had a look and I cant see an airmass composite for Meteosat anywhere which is strange

Kind Regards

Graham


EUMETSAT User Workshop and Training on Fire Monitoring

David J Taylor GM8ARV 🏴󠁧󠁢󠁳󠁣󠁴󠁿 🇪🇺
 

There's a User Workshop and Training on Fire Monitoring which may interest you.

"An online event through interactive data discovery and user practices on state-of-the-art operational datasets for detection of fire, related emissions and impacts. Jointly organized by EUMETSAT, CAMS-ECMWF, AC SAF, LSA SAF with support from Copernicus."

Details:
https://training.eumetsat.int/course/view.php?id=360

Cheers,
David
--
SatSignal Software - Quality software for you
Web: https://www.satsignal.eu
Email: david-taylor@...
Twitter: @gm8arv


Re: Pytroll/SatPy for EUMETCast

Ernst Lobsiger
 

On Thu, May 7, 2020 at 06:26 AM, Graham Woolf wrote:
Hi Ernst

I dont have an immpac folder anywhere on my C drive

Kind Regards

Graham
Graham

immpac is just my windows user name. Look where you installed miniconda3 !

C:\Users\graha\miniconda3\...

I think. You should remember your user name should you?


Ernst


Re: Pytroll/SatPy for EUMETCast

Graham Woolf
 

Hi Ernst

I dont have an immpac folder anywhere on my C drive

Kind Regards

Graham


Re: Pytroll/SatPy for EUMETCast

Ernst Lobsiger
 

Graham

You find all possible composites for all readers in.../miniconda3/.... /satpy/etc

C:\Users\immpac\miniconda3\envs\pytroll\Lib\site-packages\satpy\etc\composites

You may have to adapt the list of decompressed files for SEVIRI images in my scripts.

You find all the instrument specs here (EUMETCast normally does not send you all)

C:\Users\immpac\miniconda3\envs\pytroll\Lib\site-packages\satpy\etc\readers


Happy hacking

Ernst


P.S. This kind of stuff is for the advanced Windows Pytroll/Satpy HOWTO 2.0 (written by someone else...)


TP 1 data rate.

geojohnt@...
 

Hello All,

Does anyone know the current total and maximum possible data rate, that TP 1 can disseminate?

I saw a couple of bursts of + 74 Mbps yesterday around 12:19 UTC.

Also, my instances of 2-3 second 'bursts' of negative Mbps with the SR1 dials seems to be increasing.
Or perhaps I'm just spending too much time at the computer?

Regards,
John.

 


Re: Pytroll/SatPy for EUMETCast

Graham Woolf
 

Hi Ernst

I cant seem to find an airmass composite for the MSG satellite

I know there should be one because EUMETSAT quite often show the airmass immage on their website for example here

https://www.eumetsat.int/website/home/Images/ImageLibrary/DAT_4943581.html

I know we have an airmass composite for GOES16 so it would be good if I could find one for Meteosat

Kind Regards

Graham


Re: Pytroll/SatPy for EUMETCast

Graham Woolf
 

Hi Ernst

I seem to have all the files

I have uploaded them to here

https://drive.google.com/open?id=1OYLfUg1fdSAMUg2RN09v1e3wJopsytlp

I wonder if you could download them and see if you get the same problem

Kind Regards

Graham


Re: Pytroll/SatPy for EUMETCast

Ernst Lobsiger
 

Hi Graham

I'm quite sure now there is something wrong with your data. On my primery receiver chain
Linux --NFS--> Linux I have the FY-3D data from yesterday. I tried the daylight pass for
Switzerland and Isle of Man 20200506. It's the same orbit Nr. 12824 so the images look
the same for both POIs.  I add a sorted (!) list of the FY-3D files you should have.

You can produce this list by the following changes:

In the middle of the script:

# Sorting seems not necessary?!
# bestfiles = sorted(bestfiles)
-->
# Sorting seems not necessary?!
bestfiles = sorted(bestfiles)

At the bottom of the script:

# This is just to printout nicely
# for name in bestfiles:
#    print (name)
-->
# This is just to printout nicely
for name in bestfiles:
   print (name)

It seems your process crashes before you reach the end. So you have to copy the printout below
the sorting line above. Then you call the script: python FY-3D_VIS.py 20200506 > bestfiles.txt

Compare your list bestfiles.txt with what I attached.

Cheers
Ernst

@Douglas: Don't worry. That kind of debuging is not what you must expect on a dayly basis ...


Re: Pytroll/SatPy for EUMETCast

Ernst Lobsiger
 

Hi Graham and Jeskynar

Below the (unsorted) list of files used for the daylight pass of FY-3D over Switzerland I showed before:
...
return func(*(_execute_task(a, cache) for a in args))
Z:/E1H-RDS-2/2020/05/05\FY3D_20200505_124000_124100_12810_MERSI_1000M_L1B.HDF
Z:/E1H-RDS-2/2020/05/05\FY3D_20200505_124400_124500_12811_MERSI_1000M_L1B.HDF
Z:/E1H-RDS-2/2020/05/05\FY3D_20200505_123600_123700_12811_MERSI_GEO1K_L1B.HDF
Z:/E1H-RDS-2/2020/05/05\FY3D_20200505_123500_123600_12810_MERSI_GEO1K_L1B.HDF
Z:/E1H-RDS-2/2020/05/05\FY3D_20200505_124100_124200_12811_MERSI_GEO1K_L1B.HDF
Z:/E1H-RDS-2/2020/05/05\FY3D_20200505_124200_124300_12811_MERSI_1000M_L1B.HDF
Z:/E1H-RDS-2/2020/05/05\FY3D_20200505_124200_124300_12811_MERSI_GEO1K_L1B.HDF
Z:/E1H-RDS-2/2020/05/05\FY3D_20200505_124300_124400_12811_MERSI_1000M_L1B.HDF
Z:/E1H-RDS-2/2020/05/05\FY3D_20200505_123700_123800_12810_MERSI_1000M_L1B.HDF
Z:/E1H-RDS-2/2020/05/05\FY3D_20200505_124500_124600_12811_MERSI_GEO1K_L1B.HDF
Z:/E1H-RDS-2/2020/05/05\FY3D_20200505_123800_123900_12810_MERSI_GEO1K_L1B.HDF
Z:/E1H-RDS-2/2020/05/05\FY3D_20200505_123800_123900_12810_MERSI_1000M_L1B.HDF
Z:/E1H-RDS-2/2020/05/05\FY3D_20200505_124100_124200_12811_MERSI_1000M_L1B.HDF
Z:/E1H-RDS-2/2020/05/05\FY3D_20200505_124000_124100_12810_MERSI_GEO1K_L1B.HDF
Z:/E1H-RDS-2/2020/05/05\FY3D_20200505_124500_124600_12811_MERSI_1000M_L1B.HDF
Z:/E1H-RDS-2/2020/05/05\FY3D_20200505_123600_123700_12811_MERSI_1000M_L1B.HDF
Z:/E1H-RDS-2/2020/05/05\FY3D_20200505_124300_124400_12811_MERSI_GEO1K_L1B.HDF
Z:/E1H-RDS-2/2020/05/05\FY3D_20200505_124400_124500_12811_MERSI_GEO1K_L1B.HDF
Z:/E1H-RDS-2/2020/05/05\FY3D_20200505_123900_124000_12811_MERSI_GEO1K_L1B.HDF
Z:/E1H-RDS-2/2020/05/05\FY3D_20200505_123700_123800_12810_MERSI_GEO1K_L1B.HDF
Z:/E1H-RDS-2/2020/05/05\FY3D_20200505_123900_124000_12811_MERSI_1000M_L1B.HDF
Z:/E1H-RDS-2/2020/05/05\FY3D_20200505_123500_123600_12810_MERSI_1000M_L1B.HDF

(pytroll) C:\EMCtools\pyscripts>
...
Quite something! I do not know what happens if a couple of those files are missing. Another possibility
maybe that EUMETCast has sent all files and you have properly received them but some channel data
is marked invalid inside. Just try other dates and POI positions to see whether the problem persists.

Best Regards
Ernst


Re: Pytroll/SatPy for EUMETCast

Jeskynar
 

On Wed, 6 May 2020 at 22:31, Graham Woolf <woolfg@...> wrote:
I am having trouble with the FY3D script
It runs OK but never seems to produce any data - I have checked and I do have data files for the last two days
This is the output I am getting

The following datasets were not created and may require resampling to be generated: DatasetID(name='true_color', wavelength=None, resolution=None, polarization=None, calibration=None, level=None, modifiers=None)
 
 "No dataset matching 'DatasetID(name='true_color', wavelength=None, resolution=None, polarization=None, calibration=None, level=None, modifiers=None)' found"

The message is often given when there aren't enough of the right files for the requested output, i.e. not all of red,green,blue to make true_color so check there are no missing files