Date   

Re: RSS

David J Taylor GM8ARV ๐Ÿด๓ ง๓ ข๓ ณ๓ ฃ๓ ด๓ ฟ ๐Ÿ‡ช๐Ÿ‡บ
 

Yes you are right David, it is only affecting the meteorological
products ( with one exception.) I should have read it more carefully the
first time.

Regards
Ian.
============================

Glad you read it that way too, Ian! For one moment I thought they might have found a way of squeezing more segments into the 5 minutes, or changing the interval to 7 minutes, or something else which would have required massive amounts or reprogramming! Fortunately, not!

Cheers,
David
--
SatSignal Software - Quality software written to your requirements
Web: http://www.satsignal.eu
Email: david-taylor@...
Twitter: @gm8arv


Re: Tellicast client version 2.14.4 for Linux

Ernst Lobsiger
 

David,

This is my "understanding" so far:

When TELLICast connects to a channel it opens a socket.
Part of this system call is the size of a (net-) buffer
to be provided by the kernel. The kernel has some max
size limits for read and write buffers. It seems that
the Linux default limits are not enough for TELLICast.
In /etc/sysctl.conf a lot of different kernel defaults
can be adapted. For TC 2.14.1 EUMETSAT asked to set
maximum network buffer sizes to 5500000 Bytes. Maybe
this was a hard coded size TC 2.14.1 was asking for.
Or maybe EUMETSAT just found the clients default of
4000000 (according to TD-15) and added some slack.

Now it seems that the new TC 2.14.4 asks for 4000000 by
default but has the possibility to increase (or lower)
that with "receive_buffer_size" on a TC channel basis.
EUMETSAT now recommends receive_buffer_size=8000000
which means that the max limits in /etc/sysctl.conf
must (at least) be that size. Of course this does not
answer the question of optimum sizes per channel.

As I still had 5500000 as max limit but was asking
for 8000000 the TC client when opening the socket
got a return value that only a 5500000 Bytes buffer
size could be allocated. This was the reason for
the warning in the TELLICast clients log file.


Cheers,
Ernst

ย 


Re: RSS

Ian Deans
 

On 11/08/2018 10:11, David J Taylor via Groups.Io wrote:
I see from the Weekly Operations Schedule that there is to be an
extension of the RSS processing area from the 29/8/2018.
Unfortunately the URL they give for details and sample files does not
appear to exist !!
Regards
Ian.
=============================
Ian,
My reading of the announcement is that it only affects certain derived products, and not the basic RSS scan region itself.ย  Agreed?
Cheers,
David
==========================================================================

Yes you are right David, it is only affecting the meteorological products ( with one exception.) I should have read it more carefully the first time.

Regards
Ian.


Re: RSS

David J Taylor GM8ARV ๐Ÿด๓ ง๓ ข๓ ณ๓ ฃ๓ ด๓ ฟ ๐Ÿ‡ช๐Ÿ‡บ
 

I see from the Weekly Operations Schedule that there is to be an
extension of the RSS processing area from the 29/8/2018.

Unfortunately the URL they give for details and sample files does not
appear to exist !!

Regards
Ian.
=============================

Ian,

My reading of the announcement is that it only affects certain derived products, and not the basic RSS scan region itself. Agreed?

Cheers,
David
--
SatSignal Software - Quality software written to your requirements
Web: http://www.satsignal.eu
Email: david-taylor@...
Twitter: @gm8arv


Re: Tellicast client version 2.14.4 for Linux

David J Taylor GM8ARV ๐Ÿด๓ ง๓ ข๓ ณ๓ ฃ๓ ด๓ ฟ ๐Ÿ‡ช๐Ÿ‡บ
 

David,

Thanks for the hint with a new version of TD-15 I haven't read before.
Unfortunately the 550000 didn't ring the bell but that's what stands
in /etc/syctl.conf configuring the kernel buffer at startup so far.

The relevant text in TD-15 is:
"
receive_buffer_size=8000000
Defines the kernel receive buffer size for this multicast channel in bytes. The default
value is 4000000. Increasing the buffer helps avoiding losses if the client is
temporarily blocked in IO operation and cannot read the incoming multicast fast
enough, e.g. in case of heavy disk load. On Linux system the following system
parameters in /etc/syctl.conf must be set to the same or higher values:
net.core.rmem_max=8000000 (reboot the system after a change)
"

As I just copied the new client and adapted my cast-channels.ini files
the 550000 limit in /etc/sysctl.conf was left untouched. For the moment
there is only a *.rpm but no *.deb of the new client distributed for an
automatic installation under Debian (or Devuan that I use now). So I
adapted /etc/sysctl.conf setting read and wtite buffers to 8000000 and
rebooted/restarted with receive_buffer_size=8000000 for every channel.

Now the system accepts receive_buffer_size=8000000 without complaining.
So I can make now a more meaningfull test of TC 2.14.4 versus TC 2.24.1.


Cheers,
Ernst
=====================================

Ernst,

Glad that's no working OK.

I had already wondered whether there were similar limits on Windows - the amount of non-paged memory etc. - but these are normally set appropriately by the OS defaults, and use intervention isn't required. I do monitor the memory used by TelliCast, but it didn't seem to change much with the new version. Possibly if that buffer memory is from non-paged pool my usage monitor wouldn't detect that. Quite possibly the first you would know about it would be a crash with something like "insufficient [type of] memory available".

I still don't have a good feeling for what an appropriate value might be - whether it should be different for HVS and BS, for example, or for different channels with different data rates.

I look forward to your results!

Cheers,
David
--
SatSignal Software - Quality software written to your requirements
Web: http://www.satsignal.eu
Email: david-taylor@...
Twitter: @gm8arv


Re: Tellicast client version 2.14.4 for Linux

Ernst Lobsiger
 

David,

Thanks for the hint with a new version of TD-15 I haven't read before.
Unfortunately the 550000 didn't ring the bell but that's what stands
in /etc/syctl.conf configuring the kernel buffer at startup so far.

The relevant text in TD-15 is:
"
receive_buffer_size=8000000
Defines the kernel receive buffer size for this multicast channel in bytes. The default
value is 4000000. Increasing the buffer helps avoiding losses if the client is
temporarily blocked in IO operation and cannot read the incoming multicast fast
enough, e.g. in case of heavy disk load. On Linux system the following system
parameters in /etc/syctl.conf must be set to the same or higher values:
net.core.rmem_max=8000000 (reboot the system after a change)
"

As I just copied the new client and adapted my cast-channels.ini files
the 550000 limit in /etc/sysctl.conf was left untouched. For the moment
there is only a *.rpm but no *.deb of the new client distributed for an
automatic installation under Debian (or Devuan that I use now). So I
adapted /etc/sysctl.conf setting read and wtite buffers to 8000000 and
rebooted/restarted with receive_buffer_size=8000000 for every channel.

Now the system accepts receive_buffer_size=8000000 without complaining.
So I can make now a more meaningfull test of TC 2.14.4 versus TC 2.24.1.


Cheers,
Ernst


Re: RSS

Douglas Deans
 

On 10/08/2018 17:05, Ian Deans via Groups.Io wrote:
I see from the Weekly Operations Schedule that there is to be an extension of the RSS processing area from the 29/8/2018.
Unfortunately the URL they give for details and sample files does not appear to exist !!
Regards
Ian.
=============================================================================

There is an error in the address. I went into News Items on the Eumetsat site and found the correct address :-

https://www.eumetsat.int/website/home/News/DAT_4019034.html?lang=EN&pState=1

Regards,
Douglas.


Re: RSS

Thorsten Miglus
 

Hi Ian,

this link works for me:
https://www.eumetsat.int/website/home/News/DAT_4019034.html

Cheers,
Thorsten


On Fri, Aug 10, 2018 at 06:05 PM, Ian Deans wrote:
I see from the Weekly Operations Schedule that there is to be an extension of the RSS processing area from the 29/8/2018.

Unfortunately the URL they give for details and sample files does not appear to exist !!

Regards
Ian.


RSS

Ian Deans
 

I see from the Weekly Operations Schedule that there is to be an extension of the RSS processing area from the 29/8/2018.

Unfortunately the URL they give for details and sample files does not appear to exist !!

Regards
Ian.


Re: Tellicast client version 2.14.4 for Linux

David J Taylor GM8ARV ๐Ÿด๓ ง๓ ข๓ ณ๓ ฃ๓ ด๓ ฟ ๐Ÿ‡ช๐Ÿ‡บ
 

Hi All,

As promised I did some "beta testing" of TC client 2.14.4.
David mentionned that documentation of the new parameters
under Windows is missing. Here some answers to his questions.

I found not much new except for the EUMETSAT distributed
channel ini files do not work as expected under GNU/Linux.
"receive_buffer_size=8000000" seems to be a no go :-(.

Attached my findings so far ...

Cheers,
Ernst
==========================================

Ernst,

Many thanks for that.

As all the other parameters are specified in bytes, I expect that the buffer size is as well. The pedants would be arguing between MB and MiB!

There was a note in the current TD15 that the default buffer size - if not specified - was 4000000. That note also suggests it's since 2.14.4 (meaning 2.14.4 and later, or post 2.1.14, perhaps a subtly different meaning in the German?).

https://www.eumetsat.int/website/wcm/idc/idcplg?IdcService=GET_FILE&dDocName=PDF_TD15_EUMETCAST&RevisionSelectionMethod=LatestReleased&Rendition=Web

I found that the 2.14.4 program worked as expected both without a buffer size begin specified (i.e. just replacing the .exe and the licence.ini file in Windows systems), and with the buffer as 8000000 as in the EUMETCast example channels file.

I'm also using a RAMdisk for temp and received files on some systems, a RAMdisk for temp on others, and no RAMdisk at all on other systems (one HVS-1 and one HVS-2). I'm trying to see whether there are any differences although the measurements are made more difficult here partially because of the slight skew error, and partially because of the low signal here (with the weather conditions making things worse).

It's too early to tell whether there's any difference. Whilst I agree that having the temp.data RAMdisk helps, it's still possible that that using a bigger buffer in the TelliCast would reduce the number of calls to the OS for disk I/O operations, so there might be a benefit.

But most of my problems with HVS-2 are lost TelliCast packets, so if they're not coming in over the network there's not much you can do about that, unless in the exceptional case of the network interface (or virtual network in the TBS PCIe case) not having enough buffering. As we're using gigabit interfaces and network cards on the Ayecka SR1 that's capable of speeds some 12 times in excess of what's needed. Interestingly, the SR1 can occasionally produce zero packet loss on the PC without a RAMdisk at all, while the TBS/temp/files/RAMdisk system always has missed & recovered packets.

Fun, this TelliCast, isn't it!

Cheers,
David
--
SatSignal Software - Quality software written to your requirements
Web: http://www.satsignal.eu
Email: david-taylor@...
Twitter: @gm8arv


Re: Tellicast client version 2.14.4 for Linux

Ernst Lobsiger
 

Hi All,

As promised I did some "beta testing" of TC client 2.14.4.
David mentionned that documentation of the new parameters
under Windows is missing. Here some answers to his questions.

I found not much new except for the EUMETSAT distributed
channel ini files do not work as expected under GNU/Linux.
"receive_buffer_size=8000000" seems to be a no go :-(.

Attached my findings so far ...

Cheers,
Ernst


GOES 17 ABI problem update.

Douglas Deans
 

Another interesting update from NOAA re the GOES 17 ABI IR issue.
Looks as if it will go into GOES W service by the end of the year as even with its limitations it is still far better than GOES 15.
Please see :-

https://www.nesdis.noaa.gov/content/noaa-shares-first-infrared-imagery-goes-17-satellite

Regards,
Douglas.


Planned Termination of NOAA's U.S. DOMSAT Meteosat Second Generation (MSG) Broadcast Service

David J Taylor GM8ARV ๐Ÿด๓ ง๓ ข๓ ณ๓ ฃ๓ ด๓ ฟ ๐Ÿ‡ช๐Ÿ‡บ
 

For some USA users. Looks like Internet access will be available instead.

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
*Topic: *Planned Termination of NOAA's U.S. DOMSAT Meteosat Second Generation (MSG) Broadcast Service

*Date/Time Issued:* August 7, 2018 1430 UTC

*Product(s) or Data Impacted:* EUMETSAT's Prime Satellite (0 degree) Meteosat-11 MSG HRIT Data

*Date/Time of Initial Impact:* Tentatively mid-November 2018

*Date/Time of Expected End: *Indefinitely

*Details/Specifics of Change:* In coordination with EUMETSAT, NOAA/NESDIS plans to terminate its U.S. DOMSAT direct broadcast of MSG 0 degree Prime satellite data (currently Meteosat-11) in the mid-November 2018 time frame. This service termination is due in large part to obsolete equipment, limited distribution capacity and other factors such as the availability of robust data delivery systems.

In parallel with the U.S. DOMSAT service, NOAA/NESDIS has Meteosat-11 HRIT data available for its authorized users from its Product Distribution and Access (PDA) system (24x7 support) and from STAR FTP services (best effort support). In preparation for the loss of the U.S. DOMSAT MSG broadcast service, 24x7 operational users can access the MSG data from the NESDIS PDA or for non-operational users, from NESDIS STAR.

Please contact "nesdis.data.access@... <mailto:nesdis.data.access@...>" for additional information to acquire MSG 0 degree Prime satellite data (currently Meteosat-11) from the NESDIS PDA or from the NESDIS STAR server. Following further coordination with EUMETSAT, in the early fall 2018 time frame, NESDIS will provide the specific November 2018 date of U.S. DOMSAT MSG service termination.
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Cheers,
David
--
SatSignal Software - Quality software written to your requirements
Web: http://www.satsignal.eu
Email: david-taylor@...
Twitter: @gm8arv


Re: Tellicast client version 2.14.4 for Linux

David J Taylor GM8ARV ๐Ÿด๓ ง๓ ข๓ ณ๓ ฃ๓ ด๓ ฟ ๐Ÿ‡ช๐Ÿ‡บ
 

Hi David,

This is indeed quite an elaborate setup !
I don't monitor as such, but sometimes I leave my system running for a few days on end, and I never saw any segments loss in normal weather.
We have here in Belgium a good signal and my parabolic antenne is 1.25 m in diameter
I think that you don't need a powerful pc for Eumetcast , because the performance of the tellicast client software is very good. What's more important is the speed of the hard disks.

Cheers,
Hugo
===============================

Hugo,

The setup has just grown over the years, as I have been involved in EUMETCast almost before it started (it was used for Meteosat-6 and -7 data).

I envy your good signal!

Much of the effort in the early days (2003) was devoted to getting TelliCast to work as well as possible on the PCs which were affordable at that time. Memory was expensive, and affordable HDs were slow. Different motherboards and different network adapters performed better or worse, and sometimes you needed to find the "best" combination.

More recently the challenge has moved to receiving some 400 GB of data per day. I'd agree that for reception alone processing power is not so important, but having multiple cores helps (hyperthreading didn't work as well with earlier versions of the software). For processing both CPU power and memory is needed, particularly for the highest resolution data. Memory is also important for buffering HDs. Selection of data by PID and channel filtering is very important too.

You might like to automate missed and recovered packet reporting using one of Ernst's scripts - when you have a spare 5 minutes!

Cheers,
David
--
SatSignal Software - Quality software written to your requirements
Web: http://www.satsignal.eu
Email: david-taylor@...
Twitter: @gm8arv


Re: Tellicast client version 2.14.4 for Linux

Hugo
 

Hi David,

This is indeed quite an elaborate setup !
I don't monitor as such, but sometimes I leave my system running for a few days on end, and I never saw any segments loss in normal weather.
We have here in Belgium a good signal and my parabolic antenne is 1.25 m in diameter.
I think that you don't need a powerful pc for Eumetcast , because the performance of the tellicast client software is very good. What's more important is the speed of the hard disks.

Cheers,

Hugo


Re: Austria weather....

David J Taylor GM8ARV ๐Ÿด๓ ง๓ ข๓ ณ๓ ฃ๓ ด๓ ฟ ๐Ÿ‡ช๐Ÿ‡บ
 

Looks like storms over Aflenz causing signal level drops. Possibly a switch to the Vienna uplink instead.

Cheers,
David
===========================================

Robert Moore asked m to post this, as he is having access issues....

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
Staying with my son in South Tyrol - magnificent thunder storms, almost an hour and a half of continuous lightning last night. David's lightning detector picked up over 700 strikes within 40km (that's David Moore).
Will be posting pictures to TORRO website and a film on the web - will notify Group of the latter.
BTW South Tyrol is in Italy - used to be in Austria - but thereby hangs a complicated story!

Robert
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Cheers,
David
--
SatSignal Software - Quality software written to your requirements
Web: http://www.satsignal.eu
Email: david-taylor@...
Twitter: @gm8arv


Re: Tellicast client version 2.14.4 for Linux

David J Taylor GM8ARV ๐Ÿด๓ ง๓ ข๓ ณ๓ ฃ๓ ด๓ ฟ ๐Ÿ‡ช๐Ÿ‡บ
 

Hi David,

If you use 'minimal' hardware ( Pentium, 486 ) than Linux is the only solution. You can't run Windows 10 on a 486 PC ...
I have only one receiving PC ( a Pentium from several years ago ) were I run BS, HVS-1 and HVS-2 without any problems.
Why do you use four PC's to receive Eumetcast ? Redundency ?

Kind regards,
Hugo
========================================

For testing, Hugo, with multiple receivers, TelliCast versions, and configurations (e.g. no RAMdisk, with RAMdisk for temp, with RAMdisk for temp & received). It's actually five PCs, as shown here:

https://www.satsignal.eu/mrtg/performance_eumetcast-dvb-s2.php#config

I have a spare LNB feed I can use for checking the cross-polarised signal.

Good news that you are able to use an older PC for EUMETCast. Do you monitor and record packet loss?

BTW: I've just updated some clients to TelliCast 2.14.4 and others to add the receive_buffer parameter. I'd be interested to know what value is optimum.

Cheers,
David
--
SatSignal Software - Quality software written to your requirements
Web: http://www.satsignal.eu
Email: david-taylor@...
Twitter: @gm8arv


Re: Tellicast client version 2.14.4 for Linux

Hugo
 

Hi David,

If you use 'minimal' hardware ( Pentium, 486 ) than Linux is the only solution. You can't run Windows 10 on a 486 PC ...
I have only one receiving PC ( a Pentium from several years ago ) were I run BS, HVS-1 and HVS-2 without any problems.
Why do you use four PC's to receive Eumetcastย  ? Redundency ?

Kind regards,

Hugo


Re: Tellicast client version 2.14.4 for Linux

David J Taylor GM8ARV ๐Ÿด๓ ง๓ ข๓ ณ๓ ฃ๓ ด๓ ฟ ๐Ÿ‡ช๐Ÿ‡บ
 

Ernst,
[]
It was the first time I installed TC on Windows and the installation was going very smoothly ... Of course running TC on Windows in 'production' is not an option ....
[]
Kind regards,
Hugo
==============================

Running seven instances of TelliCast on Windows-10 in "production" on four PCs isn't an issue. Three BS, two HVS-1 and two HVS-2. The lack of a good signal is much more of a problem here!

Cheers,
David
--
SatSignal Software - Quality software written to your requirements
Web: http://www.satsignal.eu
Email: david-taylor@...
Twitter: @gm8arv


Re: Tellicast client version 2.14.4 for Linux

Hugo
 

Ernst,

Our mails crossed.
In my first attempts to install the TC client I used the installation script, but it didn't work. I saw the errors in the startup of my system ( dmesg ) and then I remembered your HOWTO and changed the 20-etoken file.
I should have going over all the possible errors in the Troubleshooting guide a bit sooner ...
Anyway, it is always an adventure installing TC on Linux.

Hugo