Date   

Re: RamDisk size for HVS-2

David J Taylor GM8ARV 🏴󠁧󠁢󠁳󠁣󠁴󠁿 🇪🇺
 

On 15/10/2021 09:18, Ernst Lobsiger via groups.io wrote:
David uses and describes a much simpler approach based on robocopy.exe and
trimtree.exe on his site but I doubt this can do what my MVMSG.cmd actually
does. I also note that he uses RAM-disk sizes up to 8GB (see e.g. his PC Lund).
Here, at the extremities of the satellite footprint, and with a dish that is
too sensitive to wind, I have much worse reception conditions than those in
central Europe!

The simple Robocopy approach works extremely well, and is much easier to
understand.

My PC Lund receives the full HVS-1 and HVS-2 on a single card:

https://www.satsignal.eu/mrtg/lund-ramdisk-trees.html

My PC Kiruna receives HVS-1 along and has no RAMdisk. Data is sent directly to
a separate 1 TB CRM hard disk.

Yes, Lund has an 8 GB RAMdisk, but you might argue that if there are such
issues that a RAMdisk that big is needed, will the data be good enough to
justify reception in the first place?

Recent experience is that when the GOES-16/17 data goes offline from the USA,
and EUMETSAT attempt to catch up, the HVS-1 data stream is overloaded in any
case. I see this both here and at Arne's.

The guidance? Get a 32 GB PC and be prepared to devote 8 GB to a RAMdisk.
Monitor it, and trim downwards as meets your particular needs.

Cheers,
David
--
SatSignal Software - Quality software for you
Web: https://www.satsignal.eu
Email: david-taylor@blueyonder.co.uk
Twitter: @gm8arv


Re: TC error messages.

Ernst Lobsiger
 

On Fri, Oct 15, 2021 at 01:34 AM, <geojohnt@...> wrote:
But do some files need to be 'renamed?'
If so, why?
John,

if your tmp and received are on the same partition rename is the normal process of "moving" a fully received tmp file to the received/channel folder. This is actually done by (fast) hard links which means no data has to be moved at all. This has all been described im my documentation for MVMSG.cmd (see my latest reply for Alan). The documentation is still in the MSG-1 file secton in the MVMSG.cmd DISTRO 022 though it's rather technical and certainly not everybody's business.

IIRC we had this message a few times and it was a transmission problem on the uplink side (not sure). On GNU/Linux you get this message for every file if tmp and received is on different file systems (partitions) and nothing works!!

Regards,
Ernst



TC error messages.

geojohnt@...
 

All,

I had an overnight error message the other day which I don't think I've seen before.

Lvl:Date       Time (UTC)  :Message
ERR:2021-10-14 00:54:13.085:Cannot rename file "\EUMETCast\tmp\bas\61677f41031979db.tmp" to "\EUMETCast\received\bas\default\Z_SATE_C_BABJ_2021101400000pO_FY2H_PRE_024_NOM.AWX.gz" (Invalid argument)
"Cannot rename file" - not that I look much at the error messages, just clear them.
But do some files need to be 'renamed?'
If so, why?

Regards,
John Tellick.









Re: RamDisk size for HVS-2

Ernst Lobsiger
 

<cite>
It's odd that on your system you never exceed 1.5GB used on the Ramdisk, unless the Windows and Linux versions of Tellicast are different in some way.
</cite>

Alan,

given your 1.8m dish and the experiments you just described you are certainly not an average Windows TelliCast user. I also had a look at your rare posts that show that you have the potential to understand and use my MVMSG.cmd. You still find the DISTRO 0.22 in the MSG-1 file section (login, go to files and search for MVMSG). The zip contains my papers with the technical background. I have written this on my Windows 7 desktop before T2 and HVS-2 even existed. It uses an NTFS-formatted RAM-disk. The reason for this are rather technical except for the fact that I had no Windows RAM-disk at all and I found it outright ridiculous to have to pay for such an ADD-on. I actually simulated a Windows 7 TelliCast receiver (something I never had and never will have) and a RAM-disk with an external ZIP-100 drive that I formatted FAT32 and NTFS.

David uses and describes a much simpler approach based on robocopy.exe and trimtree.exe on his site but I doubt this can do what my MVMSG.cmd actually does. I also note that he uses RAM-disk sizes up to 8GB (see e.g. his PC Lund).

https://www.satsignal.eu/wxsat/Ayecka/RAMdisk-HVS.html

https://www.satsignal.eu/mrtg/lund-ramdisk-trees.html

The only persons I know to have used MVMSG.cmd are well known Arne van Belle (my Beta tester for this project) and Emmanuel "Manu" Dupré (he made me brush up my frech). I also made a special version that can handle 4 TC clients. Arne used this version to demonstrate that his TBS-6983 could receive two full EUMETCast transponders by setting up a system that received 2 x T1 = 2 x (BAS + HVS-1). Before that mighty demonstration (though I always knew) it was very much under dispute and even FUD whether this novel PCIe card from a "Chinese startup" could ever receive both T1 and future T2. I just found that version MVMSG4.cmd as configured by Arne van Belle in an old e-mail. FYI I attach it to this post. MVMSG4.cmd could be used for BAS + HVS-1 + HVS-2 + future HVS-3. And you might be the person that makes it fly. AFAIK there is almost no difference between GNU/Linux and Windows versions of the TC client except for the fact that (impossible under GNU/Linux) Windows can put tmp and received files on different filesystems (partitions) what YOU SHOULD NEVER DO in the first place. Actually I see no reason why you could not run all three Windows clients on a 3-4GB RAM-disk (though I cannot test it here).

Cheers,
Ernst


P.S.
I just tested MVMSG.cmd on my Windows 10 PRO 64Bit desktop. In demo mode it still works as under Windows 7. If you really want to go for it read my documentation first and then send me a PM describing your system. I'll see what I can do.


Re: RamDisk size for HVS-2

Alan Curnow
 

Thanks Ernst for your detailed response.

I've been monitoring my HVS-2 temp files for the last six hours. When first checked there was just one 20MB file there even though it had been working for 12 hours up to that point. Over the next two hours it increased to a maximum of 21 files totalling 4.9GB. It stayed around that figure for an hour or so and then started decreasing in the number of files and memory used to a minimum of 4 files and 850MB after around 5 hours. It's now slowly increasing again.

I notice that none of the existing temp files last update status are more than 2 hours old so it seems all are being written to, and not broken, unless Tellicast is actually deleting them if they haven't been accessed in a certain time. I have a 1.8m dish in SW England and the SR1 reports a Es/No of 17.6dB. In heavy rain it has dropped to 16.9dB. The TBS5927 reports a S/N of 16.4dB and a BER of 0.000000, signal quality of 82 using IPTool v3.0.5.0

It's odd that on your system you never exceed 1.5GB used on the Ramdisk, unless the Windows and Linux versions of Tellicast are different in some way.

Checking the hvs-2 log it does report a missed/incomplete file around avery 10 mins on average.

Cheers,
Alan


Re: RamDisk size for HVS-2

Ernst Lobsiger
 

On Thu, Oct 14, 2021 at 09:31 AM, Alan Curnow wrote:
I can increase the Ramdisk to about 5.5GB but that may not be sufficient. What size Ramdisk does anyone else use for HVS-2 which works OK or am I safer using the SSD.
Alan,

this is an old question I wrote a couple of papers about. The short answer is:

I have several receivers that take BAS + HVS-1 + HVS-2 on one single 3GB RAM-Disk.

I never had a Windows receiver but this can be done on either GNU/Linux or Windows.
One of the main problem is that broken tmp files can stay for hours waiting for
retransmissions that never arrive. So if it seems you have a solution for good
weather the first rain can produce missed packets and then fill up your RAM disk.
I have my own solutions for that but probably nobody uses my Windows MVMSG.cmd.

I asked EUMETSAT long ago to add a CLI switch to the TelliCast client to instantly
delete broken tmp files. This will make handling minimal RAM disks much more easy.
Of course EUMETSAT had to discuss that with NEWTEC that holds all the source code.
According to OPS the next client will now have this (my) switch. Don't hold your breath.


Regards,
Ernst


RamDisk size for HVS-2

Alan Curnow
 

Hello,

I recently installed a second tuner (TBS5927) for HVS-2 (Sentinal OLCI and SLSSTR) to work alongside my SR1 for Basic and HVS-1.

My 500MB ramdisk was fine for Basic and HVS-1 tmp files using only around 50MB of it. I increased it to 4GB but it's still too small for HVS-2 tmp files. I've had to move HVS-2 tmp to an SSD which isn't ideal and it's currently using 4.76GB in 21 files. It seems to need several roughly 15 min bursts of S3A and then S3B data with long pauses between bursts before the tmp files are completed and get automatically deleted.

I can increase the Ramdisk to about 5.5GB but that may not be sufficient. What size Ramdisk does anyone else use for HVS-2 which works OK or am I safer using the SSD.

Cheers,
Alan


Aurora hunters look north

Ernst Lobsiger
 

Hi,

strong Aurora Borealis activity during the last few days over Iceland and the Hudson Bay. Thanks Christian for the heads up.

Cheers,
Ernst


Re: Pytroll/Satpy help on .py file

Daniele Guardigli
 

It works!!! Thank you very much Ernst for your patience


Il giorno lun 11 ott 2021 alle ore 15:53 Ernst Lobsiger via groups.io <ernst.lobsiger=belponline.ch@groups.io> ha scritto:
Daniele,


MSG seviri instrument has these channels:

# Channel | VIS006 |   HRV  | VIS008 | IR_016 | IR_039 | WV_062 |
# WaveLen |  0.635 |   0.7  |   0.81 |   1.64 |   3.90 |   6.25 |
# --------+--------+--------+--------+--------+--------+--------+
# Channel | WV_073 | IR_087 | IR_097 | IR_108 | IR_120 | IR_134 |
# WaveLen |   7.35 |   8.70 |   9.66 |  10.80 |  12.00 |  13.40 |


Satpy composite definition of 'airmass':
...
  airmass:
    compositor: !!python/name:satpy.composites.GenericCompositor
    prerequisites:
    - compositor: !!python/name:satpy.composites.DifferenceCompositor
      prerequisites:
      - wavelength: 6.2
      - wavelength: 7.3
    - compositor: !!python/name:satpy.composites.DifferenceCompositor
      prerequisites:
        - wavelength: 9.7
        - wavelength: 10.8
    - wavelength: 6.2
    standard_name: airmass
...


You need two more 'airmass' lines in your script, put it like below:

...
# We only decompress what is actually needed, see file seviri.yaml
needed_files = {'overview'         : ['VIS006', 'VIS008', 'IR_108'],
                'natural_color'    : ['VIS006', 'VIS008', 'IR_016'],
                'realistic_colors' : ['VIS006', 'VIS008', 'HRV'],
                'hrv_clouds'       : ['IR_108', 'HRV'],
                'night_fog'        : ['IR_039', 'IR_108', 'IR_120', 'IR_134'],
                'airmass'          : ['WV_062', 'WV_073', 'IR_097', 'IR_108'],
                'ir108_3d'         : ['IR_108']}
# Some composite names are too long for IM annotation left of pic
abbreviation = {'overview'         : 'overview',
                'natural_color'    : 'natural_color',
                'realistic_colors' : 'realist_color',
                'hrv_clouds'       : 'hrv_clouds',
                'night_fog'        : 'night_fog',
                'airmass'          : 'airmass',
                'ir108_3d'         : 'ir108_3d'}
...

Then set composite='airmass' and run the script ...


Hope this helps,

Ernst



--
Daniele Guardigli


Re: LaPalma eruption animation using EUMETCast data from EUMETSAT.

Ernst Lobsiger
 

Hi,

the La Palma volcano emitted a Sulfur Dioxide (SO2) plume on October 9th 2021 around 13:15 UTC. While ash shows magenta SO2 rather shows yellow.

http://5.153.116.236:85/images/Test5

Cheers,
Ernst


Re: Pytroll/Satpy help on .py file

Ernst Lobsiger
 

Daniele,


MSG seviri instrument has these channels:

# Channel | VIS006 |   HRV  | VIS008 | IR_016 | IR_039 | WV_062 |
# WaveLen |  0.635 |   0.7  |   0.81 |   1.64 |   3.90 |   6.25 |
# --------+--------+--------+--------+--------+--------+--------+
# Channel | WV_073 | IR_087 | IR_097 | IR_108 | IR_120 | IR_134 |
# WaveLen |   7.35 |   8.70 |   9.66 |  10.80 |  12.00 |  13.40 |


Satpy composite definition of 'airmass':
...
  airmass:
    compositor: !!python/name:satpy.composites.GenericCompositor
    prerequisites:
    - compositor: !!python/name:satpy.composites.DifferenceCompositor
      prerequisites:
      - wavelength: 6.2
      - wavelength: 7.3
    - compositor: !!python/name:satpy.composites.DifferenceCompositor
      prerequisites:
        - wavelength: 9.7
        - wavelength: 10.8
    - wavelength: 6.2
    standard_name: airmass
...


You need two more 'airmass' lines in your script, put it like below:

...
# We only decompress what is actually needed, see file seviri.yaml
needed_files = {'overview'         : ['VIS006', 'VIS008', 'IR_108'],
                'natural_color'    : ['VIS006', 'VIS008', 'IR_016'],
                'realistic_colors' : ['VIS006', 'VIS008', 'HRV'],
                'hrv_clouds'       : ['IR_108', 'HRV'],
                'night_fog'        : ['IR_039', 'IR_108', 'IR_120', 'IR_134'],
                'airmass'          : ['WV_062', 'WV_073', 'IR_097', 'IR_108'],
                'ir108_3d'         : ['IR_108']}
# Some composite names are too long for IM annotation left of pic
abbreviation = {'overview'         : 'overview',
                'natural_color'    : 'natural_color',
                'realistic_colors' : 'realist_color',
                'hrv_clouds'       : 'hrv_clouds',
                'night_fog'        : 'night_fog',
                'airmass'          : 'airmass',
                'ir108_3d'         : 'ir108_3d'}
...

Then set composite='airmass' and run the script ...


Hope this helps,

Ernst


Pytroll/Satpy help on .py file

Daniele Guardigli
 

Hello,

I'm sorry for my ignorance in Python language! Can anyone help me on setting 'airmass' scene on MSG4.py script? Seems it's not available but it's very useful during night and day.

Thank you!

Daniele Guardigli


Solar outage.

geojohnt@...
 

Hello All,

It's that time of year again when the Sun passes behind the satellite/s we are receiving causing a period of RF interference and loss of signal from our specific location.

I'm in the middle of the outage period here in SW London.

You can see what's going on, on an interactive map here:

https://www.satellite-calculations.com/Satellite/suninterference.php

Regards,
John Tellick.


Re: mpe

David J Taylor GM8ARV 🏴󠁧󠁢󠁳󠁣󠁴󠁿 🇪🇺
 

On 10/10/2021 08:56, Emmanuel via groups.io wrote:
Hello David
Another question: where the MSG Data Manager directory stores the NPP files
(VIIRSM and VIIRSDNB)?

I would like to display them with EumetcastView software.

Manu
It's not data I use, Manu, but there are files in MSG\Images\VIIRS

David
--
SatSignal Software - Quality software for you
Web: https://www.satsignal.eu
Email: david-taylor@blueyonder.co.uk
Twitter: @gm8arv


Re: mpe

Emmanuel
 

Hello David
 
Another question: where the MSG Data Manager directory stores the NPP files (VIIRSM and VIIRSDNB)?

I would like to display them with EumetcastView software.

Manu


Re: mpe

Emmanuel
 

The problem is solved!
In fact, I had not specified a correct directory name in Tellicast:

[channel]
name = E1B-SAF-5
target_directory = \ EUMETCast \ received \ SAF5
 
Instead of
 
[channel]
name = E1B-SAF-5
target_directory = \ EUMETCast \ received \ E1B-SAF-1
 
Indicated by David in his FAQ
 
Now the data is displayed correctly.
 
Thanks for your help!

Manu


Re: mpe

rnosborne@...
 

Manu,

I use MPE data in MSG Animator using the same software versions quoted by you and have no problems once
it is set up correctly.

As David states, the MSG data Manager does decompress the S-HSAF-H03B * .grb.gz files automatically if
the "Unzip SAF files" box is checked in the MSG Data Manager SAF setup.

Do you have "Dates in folders" checked  in the MSG Data Manager Setup because the state of this will
determine where the decompressed files are stored. Make sure that the "Dates in Folders" checkbox in the MSG Animator
setup matches that of the MSG Data Manager. I run without "Dates in folders" checked and MSG Data Manager stores the files
in <Drive Letter>:\MSG-1\Images\SAF

MSG Animator is looking for files called S-HSAF-h03B * .grb. If you decompress manually, the resultant files are called h03B * .grb
and must be renamed before MSG Animator can find them. I run without automatic decompression and have to rename the files as
part of my decompression process using 7-zip.

Also, in the MSG Animator setup, the Updates drop-down menu should be set to 5 mins. If it is left at the default value of (none), the
animation will not update automatically. Even then, I sometimes need to click the Update button on the main MSG Animator window to 
kick-start the automatic update process. Note that there is  a delay of up to 5 minutes after reception of a new file before the animation
is updated.

Regards,
Richard


Re: mpe

David J Taylor GM8ARV 🏴󠁧󠁢󠁳󠁣󠁴󠁿 🇪🇺
 

On 07/10/2021 21:23, Emmanuel via groups.io wrote:
Dear All,

I have a problem displaying MPE (precipitation) data in MSG Animator.
I do receive files of type S-HSAF-H03B * .grb.gz, MSG DATA MANAGER does delete
these files but they are not decompressed in the MSG \ images \ SAF file.
However, if I decompress the file manually and copy it to the directory, I can
clearly see the image in MSG Animator.
I have selected '' Decomp file '' SAF "" "in MSG DATA MANAGER.
I am using MSG Data MAnager version V3.1.18.1324 and MSG Animator V3.1.7.313
Do you have an idea?
Sorry for my bad English!

Manu
Manu,

Looking at the coding it should be working, but I can't verify the "un GZ" at
the moment. Perhaps you could create a simple command script to do the unzip
every 15 minutes or whatever?

I'll try to check this further, but it may be Monday before I can check this.
I think it used to work! Ah, just checked it here and I do get .GRB files, and
not the .GZ. That's using the MSG Data Manager: 3.1.17.1323, but that was just
a beta with the "formal" release.

Cheers,
David
--
SatSignal Software - Quality software for you
Web: https://www.satsignal.eu
Email: david-taylor@blueyonder.co.uk
Twitter: @gm8arv


mpe

Emmanuel
 

Dear All,

I have a problem displaying MPE (precipitation) data in MSG Animator.
I do receive files of type S-HSAF-H03B * .grb.gz, MSG DATA MANAGER does delete these files but they are not decompressed in the MSG \ images \ SAF file.
 
However, if I decompress the file manually and copy it to the directory, I can clearly see the image in MSG Animator.
 
I have selected '' Decomp file '' SAF "" "in MSG DATA MANAGER.
 
I am using MSG Data MAnager version V3.1.18.1324 and MSG Animator V3.1.7.313
 
Do you have an idea?
Sorry for my bad English!

Manu


Re: LaPalma eruption animation using EUMETCast data from EUMETSAT.

Ernst Lobsiger
 

On Tue, Sep 28, 2021 at 08:29 AM, Hugo wrote:
Hi Ernst,

Very interesting info about Webm Ernst, thanks ...

Grts,

Hugo 
Hugo,

yes, webm is the way to go. Here a last result of LaPalma eruption. Strong lava fountains with ash plumes October 4th and 5th (ash=magenta).

http://5.153.116.236:85/images/Test5

Cheers,
Ernst

1061 - 1080 of 33059