Well that is my first FY-3D image over the UK without a segment loss or partial segment loss for months although I was not able to check yesterday.
Perhaps things are being fixed now.

Regards,
Douglas

What software are you using to display it and is the data freely available on eumecast please
Brian Cook 

On Thu, Nov 5, 2020 at 05:33 AM, <brian@...> wrote:

What software are you using to display it and is the data freely available on eumecast please
Brian Cook 

Brian

FY3D is on EUMETCast HVS-1 in channel E1H-RDS-2  (part of EARS).
The image here is an Oblique Mercator projection made with a SatPy Python script.
PyTROLL/SatPy and Python is free software. There is a starter kit here:

https://groups.io/g/MSG-1/message/29367

Regards
Ernst

Note that current available xrit2pic cannot handle FY3D, at least as it is made avaiable via Eumetcast (HDF5 format). Next release wil supprt it.

Regards,
Rob.

On Tue, Nov 3, 2020 at 08:51 AM, Douglas Deans wrote:

Well that is my first FY-3D image over the UK without a segment loss or partial segment loss for months although I was not able to check yesterday.
Perhaps things are being fixed now.

Regards,
Douglas

Douglas

I checked your "yesterday" (Nov. 2nd) image and it still had parts of a segment missing.
Starting with your image (Nov. 3rd) I have now four images that seem to be as expected.

I hope EUMETSAT finally found the bug in the EARS software (no reply from Ops yet).

Regards
Ernst

On 06/11/2020 22:34, Ernst Lobsiger via groups.io wrote:

On Tue, Nov 3, 2020 at 08:51 AM, Douglas Deans wrote:
Well that is my first FY-3D image over the UK without a segment loss
or partial segment loss for months although I was not able to check
yesterday.
Perhaps things are being fixed now.
Regards,
Douglas
Douglas
I checked your "yesterday" (Nov. 2nd) image and it still had parts of a segment missing.
Starting with your image (Nov. 3rd) I have now four images that seem to be as expected.
I hope EUMETSAT finally found the bug in the EARS software (no reply from Ops yet).
Regards
Ernst

=========================================================================

Thanks Ernst. Yes I have found the same. Also NOAA 20 and NPPP (see my Twitter image yesterday) are much better although again I have not checked every day.
Yes hopefully better station co-ordination now.

Thanks for reporting.
Regards,
Douglas.

I have received this from OPS

Dear John,

We would like to apologise for the delay in getting back to you regarding the missing segments.

Our experts continue monitoring the data but they have seen an improvement in the situation since the 1^st November. Could you please confirm this is also the case for you?

 

Best regards,

 

Maria

I have replied to this confirming that I no longer see missing data.
PS Have now got Python working for GOES16/17 FY-3D SUOMI NPP and NOAA 20. Thanks very much to Ernst for the starter kit.
I am a beginner to Python but have now got well past print("Hello World")

All the best 

John

 

 

EUMETSAT User Service

On Wed, Nov 11, 2020 at 06:30 AM, <john.haslam4@...> wrote:

I have replied to this confirming that I no longer see missing data.

John and All

I just made mosaics from different passes (area Westminster) and  single passes over London (area eurol) for FY-3D, NOAA-20 and Suomi-NPP and
for the last fife days Nov. 07..11 . All in all this is 30 images from maybe 100 orbits ant tousands of segments. I have not found a single segment missing.

I got the same e-mal from OPS as John and will report my findings. I add two NOAA-20 thumbnails of my 30 images. The problem seems to be resolved.

Cheers,
Ernst

Hi Ernst

I am seeing missing segments again today on NOAA20 orbit number 15565

Are you seeing the same or is it my system just missing some files

Regards

Graham

On my system same missing segments NOAA20 orbit 15565. Night orbit 15559 ok
Both night orbit 46955 and day orbit 46962 OK for SUOMI NPP
Will check FY-3D shortly

Rgds  

John

John, Graham,

I see the same missing segments for NOAA20. For the moment only basic services, so I can't check for FY3D .

grts,

Hugo

To all

FY-3D orbit 15620 afternoon ascending over the UK no missing segments within the omerc_bb projection (roughly N Africa to Greenland)

Rgds

John

On Thu, Nov 19, 2020 at 07:51 AM, Graham Woolf wrote:

Hi Ernst

I am seeing missing segments again today on NOAA20 orbit number 15565

Are you seeing the same or is it my system just missing some files

Regards

Graham

Graham and All

I just had a look at U.K. passes of Suomi-NPP, NOAA-20, and FY-3D from the last four days.
11 passes are fine, todays NOAA-20 orbit 15565 shows the old problem again. My bet ist that
this is a software bug in the code that stitchs together the data from LANNION and SVALBARD.

NOAA-20 Orbit 15565 pass over th U.K.

-rw-r--r-- 1 root root 53224098 Nov 19 12:07 /srv/rec_0/NPP-2/SVMC_j01_d20201119_t1155229_e1156474_b15566_c20201119120600000401_eum_ops.h5
-rw-r--r-- 1 root root 51792322 Nov 19 12:09 /srv/rec_0/NPP-2/SVMC_j01_d20201119_t1156486_e1158114_b15566_c20201119120532000145_eum_ops.h5
-rw-r--r-- 1 root root 49438269 Nov 19 12:11 /srv/rec_0/NPP-2/SVMC_j01_d20201119_t1158126_e1159371_b15566_c20201119121036000176_eum_ops.h5
-rw-r--r-- 1 root root 47176429 Nov 19 12:13 /srv/rec_0/NPP-2/SVMC_j01_d20201119_t1159383_e1201029_b15566_c20201119121213000973_eum_ops.h5
-rw-r--r-- 1 root root 52510526 Nov 19 12:13 /srv/rec_0/NPP-2/SVMC_j01_d20201119_t1201041_e1202286_b15566_c20201119121221000333_eum_ops.h5

Here we go again with the same problem: All segments received but one segment pretends to have the normal length of app. 85 seconds
but regarding its size compared with adjecent segments obviously is either too short in scan lines or has many black scan lines. El
-rw-r--r-- 1 root root 34226739 Nov 19 12:22 /srv/rec_0/NPP-2/SVMC_j01_d20201119_t1202299_e1203526_b15566_c20201119121952000550_eum_ops.h5

-rw-r--r-- 1 root root 52193269 Nov 19 12:17 /srv/rec_0/NPP-2/SVMC_j01_d20201119_t1203538_e1205183_b15566_c20201119121634000711_eum_ops.h5
-rw-r--r-- 1 root root 52302947 Nov 19 12:18 /srv/rec_0/NPP-2/SVMC_j01_d20201119_t1205196_e1206441_b15566_c20201119121642000336_eum_ops.h5
-rw-r--r-- 1 root root 44722166 Nov 19 12:20 /srv/rec_0/NPP-2/SVMC_j01_d20201119_t1206453_e1208099_b15566_c20201119121649000895_eum_ops.h5
-rw-r--r-- 1 root root 34437344 Nov 19 12:21 /srv/rec_0/NPP-2/SVMC_j01_d20201119_t1208111_e1209356_b15566_c20201119122012000679_eum_ops.h5
-rw-r--r-- 1 root root 23264076 Nov 19 12:21 /srv/rec_0/NPP-2/SVMC_j01_d20201119_t1209369_e1210596_b15566_c20201119122020000870_eum_ops.h5

I'll again report to OPS but others should do as well.

Regards
Ernst

Hi Ernst

I have reported this to OPS

I am having trouble processing FY3D images though  most days when I run the script I get the following errors which dont mean much to me . I wonder if you could have a look at them

Traceback (most recent call last):
  File "c:\Users\graha\miniconda3\envs\pytroll\lib\site-packages\satpy\readers\__init__.py", line 301, in __getitem__
    return super(DatasetDict, self).__getitem__(item)
KeyError: 'true_color'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
  File "T:\PythonScripts\WindowsScripts\FY3D\FY-3D_VIS.py", line 188, in <module>
    scn.save_dataset(composite,imgdir+'/FY3D-Orbit_'+str(orbmax)+'-'+
  File "c:\Users\graha\miniconda3\envs\pytroll\lib\site-packages\satpy\scene.py", line 1302, in save_dataset
    return writer.save_dataset(self[dataset_id],
  File "c:\Users\graha\miniconda3\envs\pytroll\lib\site-packages\satpy\scene.py", line 679, in __getitem__
    return self.datasets[key]
  File "c:\Users\graha\miniconda3\envs\pytroll\lib\site-packages\satpy\readers\__init__.py", line 303, in __getitem__
    key = self.get_key(item)
  File "c:\Users\graha\miniconda3\envs\pytroll\lib\site-packages\satpy\readers\__init__.py", line 290, in get_key
    return get_key(match_key, self.keys(), num_results=num_results,
  File "c:\Users\graha\miniconda3\envs\pytroll\lib\site-packages\satpy\readers\__init__.py", line 245, in get_key
    raise KeyError("No dataset matching '{}' found".format(str(key)))
KeyError: "No dataset matching 'DatasetID(name='true_color', wavelength=None, resolution=None, polarization=None, calibration=None, level=None, modifiers=None)' found"

Now some days this will run OK but mostly it doesnt

I have tried with satpy 021 and 024 with the same problems

Regards

Graham

On 20/11/2020 11:47, Graham Woolf wrote:

Hi Ernst
I have reported this to OPS
I am having trouble processing FY3D images though  most days when I run the script I get the following errors which dont mean much to me . I wonder if you could have a look at them
Traceback (most recent call last):
  File "c:\Users\graha\miniconda3\envs\pytroll\lib\site-packages\satpy\readers\__init__.py", line 301, in __getitem__
    return super(DatasetDict, self).__getitem__(item)
KeyError: 'true_color'
During handling of the above exception, another exception occurred:
Traceback (most recent call last):
  File "T:\PythonScripts\WindowsScripts\FY3D\FY-3D_VIS.py", line 188, in <module>
    scn.save_dataset(composite,imgdir+'/FY3D-Orbit_'+str(orbmax)+'-'+
  File "c:\Users\graha\miniconda3\envs\pytroll\lib\site-packages\satpy\scene.py", line 1302, in save_dataset
    return writer.save_dataset(self[dataset_id],
  File "c:\Users\graha\miniconda3\envs\pytroll\lib\site-packages\satpy\scene.py", line 679, in __getitem__
    return self.datasets[key]
  File "c:\Users\graha\miniconda3\envs\pytroll\lib\site-packages\satpy\readers\__init__.py", line 303, in __getitem__
    key = self.get_key(item)
  File "c:\Users\graha\miniconda3\envs\pytroll\lib\site-packages\satpy\readers\__init__.py", line 290, in get_key
    return get_key(match_key, self.keys(), num_results=num_results,
  File "c:\Users\graha\miniconda3\envs\pytroll\lib\site-packages\satpy\readers\__init__.py", line 245, in get_key
    raise KeyError("No dataset matching '{}' found".format(str(key)))
KeyError: "No dataset matching 'DatasetID(name='true_color', wavelength=None, resolution=None, polarization=None, calibration=None, level=None, modifiers=None)' found"
Now some days this will run OK but mostly it doesnt
I have tried with satpy 021 and 024 with the same problems
Regards
Graham

==============================================================================================

Yes I get that quite often with FY-3D as well Graham. I have noticed that changing the lat and/or lon can sometimes resolve it so perhaps it has something to do with the tolerance of the pass seeking code.

Regards,
Douglas.

Hi Douglas

Thanks - Im glad its not just me

Do you have any lat/long values that appear to work more successfully or is it pot luck on each pass

Regards

Graham

On Fri, Nov 20, 2020 at 04:21 AM, Graham Woolf wrote:

Do you have any lat/long values that appear to work more successfully or is it pot luck on each pass

Regards

Graham

Graham and Douglas

This looks as if you do not have some data that are needed for "true-color". Maybe some VIS data is not transmitted far north in the dark?

1) Can you give me a date and POI location and range you use that shows this problem (I have data 4 days back here) ?
2) Can you try a FY-3D IR (DAY!) composite with these input data that does not produce a VIS (DAY) "true_color" image ?
3) Can you have a look at the list [bestfiles] or use good old TCLogSummary.cmd to make sure you have no missing files?

Regards
Ernst

On Fri, Nov 20, 2020 at 04:06 AM, Douglas Deans wrote:

Yes I get that quite often with FY-3D as well Graham. I have noticed that changing the lat and/or lon can sometimes resolve it so perhaps it has something to do with the tolerance of the pass seeking code.

Regards,
Douglas.

Douglas

I doubt there is a problem in my simple pass seeking code. If you change the POI this may result in another orbit number.
If you only increase search range (ran) the pass will be "longer" and maybe reach far north in the (November) winter dark.

Cheers
Ernst

Hi Ernst

Those errors were from yesterdays pass and I was using Isle of Man
# Isle of Man
lat = 54.228
lon = -4.532
ran = 20.0

Regards

Graham