Date   

Re: FY-3D.

Ernst Lobsiger
 

On Fri, Nov 20, 2020 at 04:21 AM, Graham Woolf wrote:
Do you have any lat/long values that appear to work more successfully or is it pot luck on each pass

Regards

Graham
Graham and Douglas

This looks as if you do not have some data that are needed for "true-color". Maybe some VIS data is not transmitted far north in the dark?

1) Can you give me a date and POI location and range you use that shows this problem (I have data 4 days back here) ?
2) Can you try a FY-3D IR (DAY!) composite with these input data that does not produce a VIS (DAY) "true_color" image ?
3) Can you have a look at the list [bestfiles] or use good old TCLogSummary.cmd to make sure you have no missing files?

Regards
Ernst


Re: FY-3D.

Graham Woolf
 

Hi Douglas

Thanks - Im glad its not just me

Do you have any lat/long values that appear to work more successfully or is it pot luck on each pass

Regards

Graham


Re: FY-3D.

Douglas Deans
 

On 20/11/2020 11:47, Graham Woolf wrote:
Hi Ernst
I have reported this to OPS
I am having trouble processing FY3D images though  most days when I run the script I get the following errors which dont mean much to me . I wonder if you could have a look at them
Traceback (most recent call last):
  File "c:\Users\graha\miniconda3\envs\pytroll\lib\site-packages\satpy\readers\__init__.py", line 301, in __getitem__
    return super(DatasetDict, self).__getitem__(item)
KeyError: 'true_color'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
  File "T:\PythonScripts\WindowsScripts\FY3D\FY-3D_VIS.py", line 188, in <module>
    scn.save_dataset(composite,imgdir+'/FY3D-Orbit_'+str(orbmax)+'-'+
  File "c:\Users\graha\miniconda3\envs\pytroll\lib\site-packages\satpy\scene.py", line 1302, in save_dataset
    return writer.save_dataset(self[dataset_id],
  File "c:\Users\graha\miniconda3\envs\pytroll\lib\site-packages\satpy\scene.py", line 679, in __getitem__
    return self.datasets[key]
  File "c:\Users\graha\miniconda3\envs\pytroll\lib\site-packages\satpy\readers\__init__.py", line 303, in __getitem__
    key = self.get_key(item)
  File "c:\Users\graha\miniconda3\envs\pytroll\lib\site-packages\satpy\readers\__init__.py", line 290, in get_key
    return get_key(match_key, self.keys(), num_results=num_results,
  File "c:\Users\graha\miniconda3\envs\pytroll\lib\site-packages\satpy\readers\__init__.py", line 245, in get_key
    raise KeyError("No dataset matching '{}' found".format(str(key)))
KeyError: "No dataset matching 'DatasetID(name='true_color', wavelength=None, resolution=None, polarization=None, calibration=None, level=None, modifiers=None)' found"
Now some days this will run OK but mostly it doesnt
I have tried with satpy 021 and 024 with the same problems
Regards
Graham
==============================================================================================

Yes I get that quite often with FY-3D as well Graham. I have noticed that changing the lat and/or lon can sometimes resolve it so perhaps it has something to do with the tolerance of the pass seeking code.

Regards,
Douglas.


Re: FY-3D.

Graham Woolf
 

Hi Ernst

I have reported this to OPS

I am having trouble processing FY3D images though  most days when I run the script I get the following errors which dont mean much to me . I wonder if you could have a look at them

Traceback (most recent call last):
  File "c:\Users\graha\miniconda3\envs\pytroll\lib\site-packages\satpy\readers\__init__.py", line 301, in __getitem__
    return super(DatasetDict, self).__getitem__(item)
KeyError: 'true_color'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "T:\PythonScripts\WindowsScripts\FY3D\FY-3D_VIS.py", line 188, in <module>
    scn.save_dataset(composite,imgdir+'/FY3D-Orbit_'+str(orbmax)+'-'+
  File "c:\Users\graha\miniconda3\envs\pytroll\lib\site-packages\satpy\scene.py", line 1302, in save_dataset
    return writer.save_dataset(self[dataset_id],
  File "c:\Users\graha\miniconda3\envs\pytroll\lib\site-packages\satpy\scene.py", line 679, in __getitem__
    return self.datasets[key]
  File "c:\Users\graha\miniconda3\envs\pytroll\lib\site-packages\satpy\readers\__init__.py", line 303, in __getitem__
    key = self.get_key(item)
  File "c:\Users\graha\miniconda3\envs\pytroll\lib\site-packages\satpy\readers\__init__.py", line 290, in get_key
    return get_key(match_key, self.keys(), num_results=num_results,
  File "c:\Users\graha\miniconda3\envs\pytroll\lib\site-packages\satpy\readers\__init__.py", line 245, in get_key
    raise KeyError("No dataset matching '{}' found".format(str(key)))
KeyError: "No dataset matching 'DatasetID(name='true_color', wavelength=None, resolution=None, polarization=None, calibration=None, level=None, modifiers=None)' found"

Now some days this will run OK but mostly it doesnt

I have tried with satpy 021 and 024 with the same problems

Regards

Graham


Re: FY-3D.

Ernst Lobsiger
 

On Thu, Nov 19, 2020 at 07:51 AM, Graham Woolf wrote:
Hi Ernst

I am seeing missing segments again today on NOAA20 orbit number 15565

Are you seeing the same or is it my system just missing some files

Regards

Graham
Graham and All

I just had a look at U.K. passes of Suomi-NPP, NOAA-20, and FY-3D from the last four days.
11 passes are fine, todays NOAA-20 orbit 15565 shows the old problem again. My bet ist that
this is a software bug in the code that stitchs together the data from LANNION and SVALBARD.

NOAA-20 Orbit 15565 pass over th U.K.

-rw-r--r-- 1 root root 53224098 Nov 19 12:07 /srv/rec_0/NPP-2/SVMC_j01_d20201119_t1155229_e1156474_b15566_c20201119120600000401_eum_ops.h5
-rw-r--r-- 1 root root 51792322 Nov 19 12:09 /srv/rec_0/NPP-2/SVMC_j01_d20201119_t1156486_e1158114_b15566_c20201119120532000145_eum_ops.h5
-rw-r--r-- 1 root root 49438269 Nov 19 12:11 /srv/rec_0/NPP-2/SVMC_j01_d20201119_t1158126_e1159371_b15566_c20201119121036000176_eum_ops.h5
-rw-r--r-- 1 root root 47176429 Nov 19 12:13 /srv/rec_0/NPP-2/SVMC_j01_d20201119_t1159383_e1201029_b15566_c20201119121213000973_eum_ops.h5
-rw-r--r-- 1 root root 52510526 Nov 19 12:13 /srv/rec_0/NPP-2/SVMC_j01_d20201119_t1201041_e1202286_b15566_c20201119121221000333_eum_ops.h5

Here we go again with the same problem: All segments received but one segment pretends to have the normal length of app. 85 seconds
but regarding its size compared with adjecent segments obviously is either too short in scan lines or has many black scan lines. El
-rw-r--r-- 1 root root 34226739 Nov 19 12:22 /srv/rec_0/NPP-2/SVMC_j01_d20201119_t1202299_e1203526_b15566_c20201119121952000550_eum_ops.h5

-rw-r--r-- 1 root root 52193269 Nov 19 12:17 /srv/rec_0/NPP-2/SVMC_j01_d20201119_t1203538_e1205183_b15566_c20201119121634000711_eum_ops.h5
-rw-r--r-- 1 root root 52302947 Nov 19 12:18 /srv/rec_0/NPP-2/SVMC_j01_d20201119_t1205196_e1206441_b15566_c20201119121642000336_eum_ops.h5
-rw-r--r-- 1 root root 44722166 Nov 19 12:20 /srv/rec_0/NPP-2/SVMC_j01_d20201119_t1206453_e1208099_b15566_c20201119121649000895_eum_ops.h5
-rw-r--r-- 1 root root 34437344 Nov 19 12:21 /srv/rec_0/NPP-2/SVMC_j01_d20201119_t1208111_e1209356_b15566_c20201119122012000679_eum_ops.h5
-rw-r--r-- 1 root root 23264076 Nov 19 12:21 /srv/rec_0/NPP-2/SVMC_j01_d20201119_t1209369_e1210596_b15566_c20201119122020000870_eum_ops.h5

I'll again report to OPS but others should do as well.

Regards
Ernst




Re: xrit2pic 2020.4

R. Alblas
 

Cornish Man,

Did you install the libs? decompr, hdf5, for Windows gtk20; for Pi: libs_xrit2pic, all on my website:

http://www.alblas.demon.nl/wsat/software/soft_msg.html#download2

Rob.

On 19-11-2020 20:07, Cornish Man via groups.io wrote:

Just installed it, but its missing files

 

Sent from Mail for Windows 10

 

From: R. Alblas
Sent: 19 November 2020 18:20
To: MSG-1@groups.io
Subject: [MSG-1] xrit2pic 2020.4

 

F

 



Re: xrit2pic 2020.4

Cornish Man
 

Just installed it, but its missing files

 

Sent from Mail for Windows 10

 

From: R. Alblas
Sent: 19 November 2020 18:20
To: MSG-1@groups.io
Subject: [MSG-1] xrit2pic 2020.4

 

F

 


xrit2pic 2020.4

R. Alblas
 

FYI:

xrit2pic 2020.4 is now available, for Linux, Windows and Raspberry Pi (last one only command line version).
Supports now also FY3D (just "fly-by" picture), and MSG native format. With a simple python script it is now possible to download near-realtime HRIT MSG/MSG_RSS from the Eumetsat-site, and process it using xrit2pic.
Works fine on a very old RPI (2011, I think one of the first versions).

Regards,
Rob Alblas


Re: FY-3D.

john.haslam4@...
 

To all

FY-3D orbit 15620 afternoon ascending over the UK no missing segments within the omerc_bb projection (roughly N Africa to Greenland)

Rgds

John


Re: FY-3D.

Hugo
 

John, Graham,

I see the same missing segments for NOAA20. For the moment only basic services, so I can't check for FY3D .

grts,

Hugo


Re: FY-3D.

john.haslam4@...
 

On my system same missing segments NOAA20 orbit 15565. Night orbit 15559 ok
Both night orbit 46955 and day orbit 46962 OK for SUOMI NPP
Will check FY-3D shortly

Rgds  

John


Re: FY-3D.

Graham Woolf
 

Hi Ernst

I am seeing missing segments again today on NOAA20 orbit number 15565

Are you seeing the same or is it my system just missing some files

Regards

Graham


Re: Pytroll/SatPy for EUMETCast

john.haslam4@...
 

Hi Christian

Thanks for your help. I put the following in the generic.yaml in the enhancements folder

night_test1:
    standard_name: night_test1
    operations:
    - name: inverse
      method: *inversefun
      args:
      - [true, true, true]
    - name: stretch
      method: *stretchfun
      kwargs: {stretch: linear}
    - name: gamma
      method: *gammafun
      kwargs: {gamma: 1.6}

It sucessfully created inverted images combining M15/M14/M12 for descending passes NOAA20 and SUOMI NPP today

Best Rgds

John


Re: Pytroll/SatPy for EUMETCast

Christian Peters
 

Hi John,

to skip the PaintShop pro part, put this in your personal enhancements/generic.yaml:

  night_test:
    standard_name: night_test
    operations:
      - name: inverse
        method: *inversefun
        args: [true]
      - name: stretch
        method: *stretchfun
        kwargs:
          stretch: linear

Then the inversion is done by the enhancement and you don’t have to do it in PaintShop.

Regards,

Christian 




Am 17.11.2020 um 17:19 schrieb john.haslam4 via groups.io <john.haslam4@...>:

This inspired me to create a composite night_test  which uses M12 Blue, M14 Green and M15 Red

night_test:
    compositor: !!python/name:satpy.composites.GenericCompositor
    prerequisites:
    - M15
    - M14
    - M12
    standard_name: night_test

The cold cloud tops come out dark so I inverted the image in Paint Shop to get a result similar to using Hugo's EUMETCASTView for these channels


Re: Pytroll/SatPy for EUMETCast

john.haslam4@...
 

This inspired me to create a composite night_test  which uses M12 Blue, M14 Green and M15 Red

night_test:
    compositor: !!python/name:satpy.composites.GenericCompositor
    prerequisites:
    - M15
    - M14
    - M12
    standard_name: night_test

The cold cloud tops come out dark so I inverted the image in Paint Shop to get a result similar to using Hugo's EUMETCASTView for these channels


Re: Pytroll/SatPy for EUMETCast

Ferdinand Valk
 

Thanks Ernst, good gardening!

Winter?  Something long forgotten here.....

 

From: MSG-1@groups.io On Behalf Of Ernst Lobsiger via groups.io
Sent: Monday, 16 November, 2020 12:49
To: MSG-1@groups.io
Subject: Re: [MSG-1] Pytroll/SatPy for EUMETCast

 

Ferdinand,

I'am really a scientist but I have worked as a teacher most of the time.
Therefore it's nice to see you arrived at chapter 8.) of my text already:

8) Where do we go from here
-------------------------------------
...
What composites need what instrument channels? Find answers here:
C:\users\username\miniconda3\envs\pytroll\Lib\site-packages\satpy\etc\composites
...
For almost every RGB composite there are interesting short "Guides" like the one I attached.
You find them all in directory "Guides" of Diego Souzas SHOWCast program suite which under
the hood actually uses Pytroll/Satpy for image processing. You find SHOWCast here:

https://geonetcast.wordpress.com/2020/07/08/new-showcast-release-v-2-0-0/

My 2.0 version is still on the TODO list. I have to ready my garden (jungle) for the looming winter.

Best regards
Ernst


Re: Pytroll/SatPy for EUMETCast

Ernst Lobsiger
 

Ferdinand,

I'am really a scientist but I have worked as a teacher most of the time.
Therefore it's nice to see you arrived at chapter 8.) of my text already:

8) Where do we go from here
-------------------------------------
...
What composites need what instrument channels? Find answers here:
C:\users\username\miniconda3\envs\pytroll\Lib\site-packages\satpy\etc\composites
...
For almost every RGB composite there are interesting short "Guides" like the one I attached.
You find them all in directory "Guides" of Diego Souzas SHOWCast program suite which under
the hood actually uses Pytroll/Satpy for image processing. You find SHOWCast here:

https://geonetcast.wordpress.com/2020/07/08/new-showcast-release-v-2-0-0/

My 2.0 version is still on the TODO list. I have to ready my garden (jungle) for the looming winter.

Best regards
Ernst


Re: Pytroll/SatPy for EUMETCast

Ferdinand Valk
 

Ernst,

 

If you are planning to release a version 2 of the HOWTO for Windows you may consider to include the attached file, which contains the various channel combinations to produce composites for the Goes satellites. It can be inserted in your goes16.py and goes17.py scripts and works for both full disk and field-of-view defined region.

 

Cheers,

Ferdinand


Re: FY-3D.

Ernst Lobsiger
 

On Wed, Nov 11, 2020 at 06:30 AM, <john.haslam4@...> wrote:
I have replied to this confirming that I no longer see missing data.
John and All

I just made mosaics from different passes (area Westminster) and  single passes over London (area eurol) for FY-3D, NOAA-20 and Suomi-NPP and
for the last fife days Nov. 07..11 . All in all this is 30 images from maybe 100 orbits ant tousands of segments. I have not found a single segment missing.

I got the same e-mal from OPS as John and will report my findings. I add two NOAA-20 thumbnails of my 30 images. The problem seems to be resolved.

Cheers,
Ernst


Re: FY-3D.

john.haslam4@...
 

I have received this from OPS

Dear John,

We would like to apologise for the delay in getting back to you regarding the missing segments.

Our experts continue monitoring the data but they have seen an improvement in the situation since the 1st November. Could you please confirm this is also the case for you?

 

Best regards,

 

Maria

I have replied to this confirming that I no longer see missing data.
PS Have now got Python working for GOES16/17 FY-3D SUOMI NPP and NOAA 20. Thanks very much to Ernst for the starter kit.
I am a beginner to Python but have now got well past print("Hello World")

All the best 

John

 

 

EUMETSAT User Service

1161 - 1180 of 31368