Date   
Re: Pytroll/SatPy for EUMETCast

Christian Peters
 

Ernst,

indeed...I got the sam problem as I switched my MSG script to a more recent env with a more recent satpy version.
So I asked at Slack whether it's a known problem and Simon answered:
"There's been a few times when I've found things that used to work but do not work any more. reduce_data=False fixes about 80% of them in my experience!"

:-D

So I just give it a try and it works....
But I think someone of the developers has to look into that ...! ;-)

Regards,

Christian

Am 22.05.20 um 15:21 schrieb Ernst Lobsiger via groups.io:

On Fri, May 22, 2020 at 03:27 AM, Christian Peters wrote:
if you still struggle with the AssertionError, you could try adding

reduce_data=False to the resample call, like

local_scene = global_scene.resample(area, reduce_data=False, radius_of_influence = 20000)
Christian,

I tried it here under Windows 10 and it works! Is this a work-around you got from slack?

Best regards
Ernst

Re: Some thoughts towards the future....

geojohnt@...
 

Hello Ernst,

Gosh, that is impressive.
Is it 15 computers?

I live around 20 km from Heathrow Airport on the east side - one of Europe's busiest airports.
Very few planes flying and very little noise.
Add to that very little blown in pollution from the airport + very little local traffic pollution - clean air for all!

No wonder the birds are singing so loudly.

Best wishes,
John.

+++++++++++++++++  


-----Original Message-----
From: Ernst Lobsiger via groups.io <ernst.lobsiger@...>
To: MSG-1@groups.io
Sent: Mon, 18 May 2020 22:06
Subject: Re: [MSG-1] Some thoughts towards the future....

On Mon, May 18, 2020 at 12:26 PM, <geojohnt@...> wrote:
Add to that Metop 2nd generation and ones going to need a super-computer - and an awful lot of time to actually sort through all the data/imaging.
John,

that's a minor problem. Here is "SuperTux 3", the third Beowulf Cluster I built.
Used for aircraft noise calculations so far. But with Corona no aircraft traffic
means no noise,no contracts. So it's free for MTG-1,  Metop 2nd generation ...

Cheers,
Ernst
_._,_._,_


Re: Some thoughts towards the future....

Ernst Lobsiger
 

On Fri, May 22, 2020 at 07:21 AM, Graham Woolf wrote:
I would quite like to have a go
Graham,

I did it under GNU/Linux with some Quick&Dirty changes to an advanced msg4.py script.
Your fastest track under Windows 10 is to start with a working copy of your msg4.py script.

Download a full uncompressed 10 Minutes RepeatCicle (RC) of test data here (RC 72 is about 12:00)

ftp://ftp.eumetsat.int/pub/OPS/out/test-data/Test-data-for-External-Users/MTG_FCI_Test-Data/FCI_L1C_24hr_Test_Data_for_Users/1.1/UNCOMPRESSED/

Take this RC directory as segdir. With this directory you directly make a "files = glob(segdir  +  "/*")  to get all these *.nc in the filelist files[].

global_scene = Scene(filenames = files, reader = 'fci_l1c_fdhsi')

Instead of IM "-rotate 180" you have to use "-flip -equalize". Not sure IM under Windows 10 will do that: I had to allow IM for more
memory usage under GNU/Linux. In any case you should get the (flipped) *.png without annotation in the respective tmp directory.

Good Luck
Ernst

Re: Some thoughts towards the future....

Graham Woolf
 

Hi Ernst

Where can I find the link for the python scripts you used for MTG-1

I would quite like to have a go

Kind Regards

Graham

Re: Pytroll/SatPy for EUMETCast

Ernst Lobsiger
 

On Fri, May 22, 2020 at 03:27 AM, Christian Peters wrote:
if you still struggle with the AssertionError, you could try adding

reduce_data=False to the resample call, like

local_scene = global_scene.resample(area, reduce_data=False, radius_of_influence = 20000)
Christian,

I tried it here under Windows 10 and it works! Is this a work-around you got from slack?

Best regards
Ernst

Re: Pytroll/SatPy for EUMETCast

Graham Woolf
 

Hi Christian

Thanks I'll give it a try

Kind Regards

Graham

Re: Pytroll/SatPy for EUMETCast

Christian Peters
 

Graham,

if you still struggle with the AssertionError, you could try adding

reduce_data=False to the resample call, like

local_scene = global_scene.resample(area, reduce_data=False, radius_of_influence = 20000)

Regards,

Christian


Am 17.05.20 um 16:33 schrieb Graham Woolf:

On Sun, May 17, 2020 at 05:40 AM, Ernst Lobsiger wrote:
'seviri_0deg',
Hi Ernst

Thanks  - I have tried your suggestion but dont get an image

This is my code

# Now all the files needed should be decompressed in tmpdir
files = glob(tmpdir + "/" + "H-000-MSG4*" + Dat + "*")
global_scene = Scene(filenames = files, reader = 'seviri_l1b_hrit')
global_scene.load([composite])

area = 'seviri_0deg'

local_scene = global_scene.resample(area, radius_of_influence = 20000)


local_scene.save_dataset(composite,imgdir + "/" + "MSG4.png",
    overlay = {'coast_dir': "C:\Tools\gshhs",
    'color': (255, 255, 0), 'width': 1.0, 'resolution': 'i'})


I get the following output

 proj_string = self.to_proj4()
Traceback (most recent call last):
  File "T:\PythonScripts\WindowsScripts\MSG\MSG4test.py", line 88, in <module>
    local_scene = global_scene.resample(area, radius_of_influence = 20000)
  File "c:\Users\graha\miniconda3\envs\pytroll\lib\site-packages\satpy\scene.py", line 1111, in resample
    self._resampled_scene(new_scn, destination, resampler=resampler,
  File "c:\Users\graha\miniconda3\envs\pytroll\lib\site-packages\satpy\scene.py", line 1055, in _resampled_scene
    dataset = self._slice_data(source_area, (slice_x, slice_y), dataset)
  File "c:\Users\graha\miniconda3\envs\pytroll\lib\site-packages\satpy\scene.py", line 992, in _slice_data
    assert ('x', source_area.x_size) in dataset.sizes.items()
AssertionError

Kind Regards

Graham

Re: Land mask for HRPT Reader

David J Taylor GM8ARV 🏴󠁧󠁢󠁳󠁣󠁴󠁿 🇪🇺
 

Hello to the group members

Did any of you tried to use land mask in HRPT Reader for visualizing SST?
Because i want to use land mask but i couldn't find the appropriate land mask data file in the internet.

Best Regards
Mr. Hamdan
======================================

I have provided a copy of the data here:

https://www.satsignal.eu/software/OdpsLandMask.zip

The original source no longer lists it.

Cheers,
David
--
SatSignal Software - Quality software for you
Web: https://www.satsignal.eu
Email: david-taylor@...
Twitter: @gm8arv

Land mask for HRPT Reader

Hamdan
 

Hello to the group members 

Did any of you tried to use land mask in HRPT Reader for visualizing SST?
Because i want to use land mask but i couldn't find the appropriate land mask data file in the internet.

Best Regards
Mr. Hamdan 

Re: msg animator setup for Hurricanes

Thorsten Miglus
 

Hi Roland,

you can change within "Position and size" the values "Left" and "Top" to define the upper left corner of your animation.
The values for "Width" and "Height" should be smaller than the pixels on you screen.
Or you use the "Define area…" button. Select a area smaller than your screen.
I can not see any problems with tihis settings.

Cheers,
Thorsten


On Wed, May 20, 2020 at 11:27 AM, fly.fechter@... wrote:
Hallo,
I am unable to setup my animator for receiving Hurricane images via Goes 16.
I can not change the area covered which stays stuck somewhere at around 20N 110W
Could you please advise what to do ?
Thanks in advance,
Regards,
Roland 

Re: Pytroll/SatPy for Eumetcast.

Douglas Deans
 

On 20/05/2020 18:28, Christian Peters via groups.io wrote:
Douglas,
it's a wide field to play. You will have lot's of fun.
Try "ir108_3d" composite with channel 10.8 and you got a nice b/w picture with white clouds without IM inverse nedde...! ;-)
Regards,
Christian
============================================================

Excellent. Will certainly give that a try. Many thanks Christian.

Kind regards,
Douglas.

Re: Pytroll/SatPy for Eumetcast.

Christian Peters
 

Douglas,

it's a wide field to play. You will have lot's of fun.

Try "ir108_3d" composite with channel 10.8 and you got a nice b/w picture with white clouds without IM inverse nedde...! ;-)

Regards,

Christian

Am 20.05.20 um 16:18 schrieb Douglas Deans via groups.io:

On 20/05/2020 14:51, Ernst Lobsiger via groups.io wrote:
Douglas,

You actually published the relevant script part some posts back:

...
# Now all the files needed should be decompressed in tmpdir
files = glob(tmpdir + "/" + "H-000-MSG4*" + Dat + "*")
global_scene = Scene(filenames = files, reader = 'seviri_l1b_hrit')

# This image has South on top, image size is 3712x3712 pixels
global_scene.save_dataset(composite,'./MSG4.png',
overlay = {'coast_dir': 'H:/EMCdata/gshhg-shp',
'color': (255, 255, 0), 'width': 1.0, 'resolution': 'i'})
...

Sorry I did not see that the  load() line was missing.

One problem of Python is that it often is very noisy with warnings
and sometimes errors that come from the underlying Satpy and
other modules only very few people really understand. Neither
Christian nor me are real Python programmers. The good side
is that you do not have to know very much to get it running.
Then you experiment and learn by doing ...

Regards,
Ernst
==============================================================

No problems Ernst. I had been struggling with that for a few days before seeking help but that is one line I won't forget again.

Tried an IR (Channel 9, 10.8 µm) successfully and even managed to find out how to make a negative (white cold) with IM (  -negate).

Many thanks again.

Best regards,
Douglas.

Re: Pytroll/SatPy for Eumetcast.

Douglas Deans
 

On 20/05/2020 14:51, Ernst Lobsiger via groups.io wrote:
Douglas,
You actually published the relevant script part some posts back:
...
# Now all the files needed should be decompressed in tmpdir
files = glob(tmpdir + "/" + "H-000-MSG4*" + Dat + "*")
global_scene = Scene(filenames = files, reader = 'seviri_l1b_hrit')
# This image has South on top, image size is 3712x3712 pixels
global_scene.save_dataset(composite,'./MSG4.png',
overlay = {'coast_dir': 'H:/EMCdata/gshhg-shp',
'color': (255, 255, 0), 'width': 1.0, 'resolution': 'i'})
...
Sorry I did not see that the  load() line was missing.
One problem of Python is that it often is very noisy with warnings
and sometimes errors that come from the underlying Satpy and
other modules only very few people really understand. Neither
Christian nor me are real Python programmers. The good side
is that you do not have to know very much to get it running.
Then you experiment and learn by doing ...
Regards,
Ernst
==============================================================

No problems Ernst. I had been struggling with that for a few days before seeking help but that is one line I won't forget again.

Tried an IR (Channel 9, 10.8 µm) successfully and even managed to find out how to make a negative (white cold) with IM ( -negate).

Many thanks again.

Best regards,
Douglas.

Re: Pytroll/SatPy for Eumetcast.

Ernst Lobsiger
 

Douglas,

You actually published the relevant script part some posts back:

...
# Now all the files needed should be decompressed in tmpdir
files = glob(tmpdir + "/" + "H-000-MSG4*" + Dat + "*")
global_scene = Scene(filenames = files, reader = 'seviri_l1b_hrit')

# This image has South on top, image size is 3712x3712 pixels
global_scene.save_dataset(composite,'./MSG4.png',
overlay = {'coast_dir': 'H:/EMCdata/gshhg-shp',
'color': (255, 255, 0), 'width': 1.0, 'resolution': 'i'})
...

Sorry I did not see that the  load() line was missing.

One problem of Python is that it often is very noisy with warnings
and sometimes errors that come from the underlying Satpy and
other modules only very few people really understand. Neither
Christian nor me are real Python programmers. The good side
is that you do not have to know very much to get it running.
Then you experiment and learn by doing ...

Regards,
Ernst

Re: Pytroll/SatPy for Eumetcast.

Ernst Lobsiger
 

On Wed, May 20, 2020 at 03:46 AM, Douglas Deans wrote:
global_scene.load([composite])
Douglas,

this line is always necessary. The order is:

a) Ready (e.g. decompress, etc ..) the files needed for a certain composite
b) Define the scene with the filelist for that composite and the appropriate reader
c) Tell the scene to load the composite (your missing line that made all the difference)
d) Resample for a certain area projection or if channels have different resolutions
e) Save to file adding coastlines (grids and decorates possible, *.png or *.jpg possible)
f) I annotate with Image Magick (outside image possible, not possible with decorate)

If you get stuck again, feel free to ask and just attach your script as *.zip.

Regards
Ernst


Pytroll/SatPy for Eumetcast.

Douglas Deans
 

Sorted !
Now working for the fulldisc.
Needed to add the line :-

global_scene.load([composite])

although I am not sure why it was not there.

Anyway many thanks Ernst for all your efforts. Good learning curve.
Will leave you in peace now.
Hope to slowly get myself on top of this during the coming weeks/months.

Thanks also to you Graham for your help.

Kind regards,
Douglas.

Re: Pytroll/SatPy for Eumetcast.

Ernst Lobsiger
 

On Wed, May 20, 2020 at 02:15 AM, Douglas Deans wrote:
Ah thanks Graham. That is what I have been using all along without success.
Just updated pytroll and pyresample (did not need done) but no change.
Douglas,
really strange, it works for Graham and me but not for you. There is one more workaround you can try:

composite = 0.635
needed_files = ['VIS006']

but then you have to change at the bottom of the script:

 imgdir + "/" + "MSG4-" + Dat + "-" + composite + ".jpg"

to
 
 imgdir + "/" + "MSG4-" + Dat + "-" + str(composite) + ".jpg"


We have to change  the floating point 0.635 to a string '0.635' to concatenate with other strings.
This works here as well.

Regards,
Ernst

Re: Pytroll/SatPy for Eumetcast.

Douglas Deans
 

On 20/05/2020 10:15, Douglas Deans via groups.io wrote:
On 20/05/2020 10:09, Graham Woolf wrote:
Hi Douglas

All I did was change this

composite = 'VIS006'
needed_files = ['VIS006']

Regards

Graham
===================================================
Ah thanks Graham. That is what I have been using all along without success.
Just updated pytroll and pyresample (did not need done) but no change.
I'll progress with the other things Ernst ask me to look at.
Regards,
Douglas.
===========================================================

Further progress. It works with the msg4.westminster.py script.

Regards,
Douglas.

msg animator setup for Hurricanes

fly.fechter@t-online.de
 

Hallo,
I am unable to setup my animator for receiving Hurricane images via Goes 16.
I can not change the area covered which stays stuck somewhere at around 20N 110W
Could you please advise what to do ?
Thanks in advance,
Regards,
Roland 

Re: Pytroll/SatPy for Eumetcast.

Douglas Deans
 

On 20/05/2020 10:09, Graham Woolf wrote:
Hi Douglas
All I did was change this
composite = 'VIS006'
needed_files = ['VIS006']
Regards
Graham
===================================================

Ah thanks Graham. That is what I have been using all along without success.
Just updated pytroll and pyresample (did not need done) but no change.

I'll progress with the other things Ernst ask me to look at.

Regards,
Douglas.