Missing packets


Daniele Guardigli
 

Hi to all,

I'm experiencing a lot of missed packets like never before and I didn't change anything in my configuration. The setup is made up of Laptop with i7 2.2Ghz, 16GB Ram, TBS5927, hub usb with 1tb ext hdd and tellicast eku.
Thanks.


Daniele Guardigli


David J Taylor GM8ARV 🏴󠁧󠁢󠁳󠁣󠁴󠁿 🇪🇺
 

On 30/05/2021 08:29, Daniele Guardigli wrote:
Hi to all,
I'm experiencing a lot of missed packets like never before and I didn't change anything in my configuration. The setup is made up of Laptop with i7 2.2Ghz, 16GB Ram, TBS5927, hub usb with 1tb ext hdd and tellicast eku.
Thanks.
Daniele Guardigli
Daniele,

What is your link margin? Perhaps the antenna alignment has slipped?

Cheers,
David
--
SatSignal Software - Quality software for you
Web: https://www.satsignal.eu
Email: david-taylor@blueyonder.co.uk
Twitter: @gm8arv


Daniele Guardigli
 

Sorry David,

what is the link margin? I'm a complete dummy :-) 

Anyway attached there's my TBS screenshot.

Daniele
Immagine 2021-05-30 095114.jpg

Il giorno dom 30 mag 2021 alle ore 09:38 David J Taylor GM8ARV 🏴󠁧󠁢󠁳󠁣󠁴󠁿 🇪🇺 via groups.io <david-taylor=blueyonder.co.uk@groups.io> ha scritto:
On 30/05/2021 08:29, Daniele Guardigli wrote:
> Hi to all,
>
> I'm experiencing a lot of missed packets like never before and I didn't
> change anything in my configuration. The setup is made up of Laptop with
> i7 2.2Ghz, 16GB Ram, TBS5927, hub usb with 1tb ext hdd and tellicast eku.
> Thanks.
>
>
> Daniele Guardigli

Daniele,

What is your link margin?  Perhaps the antenna alignment has slipped?

Cheers,
David
--
SatSignal Software - Quality software for you
Web: https://www.satsignal.eu
Email: david-taylor@...
Twitter: @gm8arv







--
Daniele Guardigli


Ernst Lobsiger
 

On Sun, May 30, 2021 at 12:51 AM, Daniele Guardigli wrote:
what is the link margin? I'm a complete dummy :-) 
 
Daniele,

Your SNR is 12.7 dB. HVS-1 should work with missed packets then and when. For Link Margin (LM) see:

https://groups.io/g/MSG-1/message/31306

Regards,
Ernst


Daniele Guardigli
 

Ok I understand, regarding hvs-1, but the issue I wrote before was on bas. The link margin should be enough for that


Il Dom 30 Mag 2021, 10:10 Ernst Lobsiger via groups.io <ernst.lobsiger=belponline.ch@groups.io> ha scritto:
On Sun, May 30, 2021 at 12:51 AM, Daniele Guardigli wrote:
what is the link margin? I'm a complete dummy :-) 
 
Daniele,

Your SNR is 12.7 dB. HVS-1 should work with missed packets then and when. For Link Margin (LM) see:

https://groups.io/g/MSG-1/message/31306

Regards,
Ernst


Ernst Lobsiger
 

Daniele,

if you still have the packet loss problem on BAS with SNR=12.7dB then all we can say it's not because of low link margin.
The next culprit often is your harddisk especially if used by other apps. Are you sure your HDD is using CMR recording?
What exactly is the model of your HDD? If the problem is on BAS only while HVS-1 has no problem: Do you use different
HDDs or disk partitions for BAS and HVS-1? For CMR versus SMR recording search this list ...

Regards,
Ernst

P.S. It's also good practice to check the logs of your TC clients. Something unusual found in there?


Ernst Lobsiger
 

On Sun, May 30, 2021 at 12:30 AM, Daniele Guardigli wrote:
I'm experiencing a lot of missed packets like never before and I didn't change anything in my configuration. The setup is made up of Laptop with i7 2.2Ghz, 16GB Ram, TBS5927, hub usb with 1tb ext hdd and tellicast eku.
Thanks.
 
 
Daniele Guardigli
Daniele,

from your first post it even seems you use an external usb hub with a 1GB external HDD attached. If your TelliCast client file system directly goes to this HDD then I'd suspect the USB system (with Receiver, EKU, HDD is kind of overloaded). If the system ran fine for some time but then suddenly almost stopped working, then again check your external HDD for CMR versus SMR recording. It *MUST* be CMR. Unfortunately not all manufacturers specify that. If your HDD has a large buffer (disk cache memory) like 128 - 256MB then this points to SMR.

Ernst


Daniele Guardigli
 

Ok, at the moment I'm trying to set tmp folders on my internal ssd and final folders on ext hdd. In your opinion is it ok?


Il giorno dom 30 mag 2021 alle ore 11:57 Ernst Lobsiger via groups.io <ernst.lobsiger=belponline.ch@groups.io> ha scritto:
On Sun, May 30, 2021 at 12:30 AM, Daniele Guardigli wrote:
I'm experiencing a lot of missed packets like never before and I didn't change anything in my configuration. The setup is made up of Laptop with i7 2.2Ghz, 16GB Ram, TBS5927, hub usb with 1tb ext hdd and tellicast eku.
Thanks.
 
 
Daniele Guardigli
Daniele,

from your first post it even seems you use an external usb hub with a 1GB external HDD attached. If your TelliCast client file system directly goes to this HDD then I'd suspect the USB system (with Receiver, EKU, HDD is kind of overloaded). If the system ran fine for some time but then suddenly almost stopped working, then again check your external HDD for CMR versus SMR recording. It *MUST* be CMR. Unfortunately not all manufacturers specify that. If your HDD has a large buffer (disk cache memory) like 128 - 256MB then this points to SMR.

Ernst



--
Daniele Guardigli


Ernst Lobsiger
 

On Mon, May 31, 2021 at 07:12 AM, Daniele Guardigli wrote:
Ok, at the moment I'm trying to set tmp folders on my internal ssd and final folders on ext hdd. In your opinion is it ok?
Daniele,

this will not help if your external disk uses SMR recording. After while the external disk gets full and files have to be deleted and overwritten.
If the disk uses SMR recording you end up with the same problem. You should still find out whether your external disk uses SMR or CMR. If it's
a  2 1/2 inch disk and has 128 - 256MB buffers it's almost 100% SMR. What you can do is write BOTH tmp and received to the SSD
for a while just to check whether you see a big improvement of reception quality (much less or no missed packets at all!). Then you
know for almost sure that your external disk is crap. Another idea is to reformat your external disk and start with a clean file system.
Then it should work for a while until you reach the point where raw received files have to be deleted and overwritten.

Regards,
Ernst


David J Taylor GM8ARV 🏴󠁧󠁢󠁳󠁣󠁴󠁿 🇪🇺
 

On 31/05/2021 15:12, Daniele Guardigli wrote:
Ok, at the moment I'm trying to set tmp folders on my internal ssd and final folders on ext hdd. In your opinion is it ok?
Daniele,

A number of us use a RAMdisk for temp, and sometimes final directories as well. I use RoboCopy (Windows) to move from the RAMdisk to HD every minute. On BAS systems here I have one 2 GB RAMdisk and one 4 GB RAMdisk used in this way.

Cheers,
David
--
SatSignal Software - Quality software for you
Web: https://www.satsignal.eu
Email: david-taylor@blueyonder.co.uk
Twitter: @gm8arv


Ernst Lobsiger
 

On Mon, May 31, 2021 at 07:54 AM, David J Taylor GM8ARV 🏴󠁧󠁢󠁳󠁣󠁴󠁿 🇪🇺 wrote:
A number of us use a RAMdisk for temp, and sometimes final directories as well. I use RoboCopy (Windows) to move from the RAMdisk to HD every minute. On BAS systems here I have one 2 GB RAMdisk and one 4 GB RAMdisk used in this way.
David and Daniele,

Even a RAM-disk is of no use if you copy the files from there onto a HDD with SMR recording. I tried this setup with a final 2 1/2'' WD Blue 1TB (SMR) and it didn't work either ...

Ernst


David J Taylor GM8ARV 🏴󠁧󠁢󠁳󠁣󠁴󠁿 🇪🇺
 

On 31/05/2021 16:31, Ernst Lobsiger via groups.io wrote:
David and Daniele,
Even a RAM-disk is of no use if you copy the files from there onto a HDD with SMR recording. I tried this setup with a final 2 1/2'' WD Blue 1TB (SMR) and it didn't work either ...
Ernst
Ernst,

I've never determined what my HDDs are, and I've had none of the issues you mention. On my BAS system Harstad the files are copied using Robocopy to a 1 TB HD (Western Digital Blue 3.5"), and then over the house gigabit network for processing on another PC.

That same PC takes a limited subset of the HVS-1 stream and writes it to another WD Blue 3.5" drive, with no RAMdisk, and no errors.

Might there be a difference when using Windows rather than Linux? Yes, it would surprise me as well, but I can't relate to your experiences with HD recording methods. Or 2.5" versus 3.5" disks?

Cheers,
David
--
SatSignal Software - Quality software for you
Web: https://www.satsignal.eu
Email: david-taylor@blueyonder.co.uk
Twitter: @gm8arv


Ernst Lobsiger
 

On Mon, May 31, 2021 at 11:35 AM, David J Taylor GM8ARV 🏴󠁧󠁢󠁳󠁣󠁴󠁿 🇪🇺 wrote:
Might there be a difference when using Windows rather than Linux? Yes, it would surprise me as well, but I can't relate to your experiences with HD recording methods. Or 2.5" versus 3.5" disks?
David,

This SMR crap is rather recent. The WD Blue 1TB 3.5'' is still using CMR recording. I run a couple of those with no problem. On the other hand bigger 2.5'' HDDs are mostly with SMR recording.
The bottom line is: Use CMR for many fast write/read/delete operations, use SMR for write once / read many times (archives, films, impossible for EUMETCast raw data because once the disk is
more or less full data must be deleted and overwritten with the speed of the incomming stream. But SMR cannot cope with that.). This has nothing to do with the OS Windows versus GNU/Linux.

Regards,
Ernst


Daniele Guardigli
 

I tried my configuration with temp folders in laptop ssd drive and final folders in external hdd, all the afternoon. It seems...that works! I'll test it during the next days.


Il Lun 31 Mag 2021, 21:13 Ernst Lobsiger via groups.io <ernst.lobsiger=belponline.ch@groups.io> ha scritto:
On Mon, May 31, 2021 at 11:35 AM, David J Taylor GM8ARV 🏴󠁧󠁢󠁳󠁣󠁴󠁿 🇪🇺 wrote:
Might there be a difference when using Windows rather than Linux? Yes, it would surprise me as well, but I can't relate to your experiences with HD recording methods. Or 2.5" versus 3.5" disks?
David,

This SMR crap is rather recent. The WD Blue 1TB 3.5'' is still using CMR recording. I run a couple of those with no problem. On the other hand bigger 2.5'' HDDs are mostly with SMR recording.
The bottom line is: Use CMR for many fast write/read/delete operations, use SMR for write once / read many times (archives, films, impossible for EUMETCast raw data because once the disk is
more or less full data must be deleted and overwritten with the speed of the incomming stream. But SMR cannot cope with that.). This has nothing to do with the OS Windows versus GNU/Linux.

Regards,
Ernst


Ernst Lobsiger
 

On Mon, May 31, 2021 at 12:15 PM, Daniele Guardigli wrote:
I'll test it during the next days.
Daniele,

if you start with a clean disk, the traffic jam can suddenly build up days later.

https://groups.io/g/MSG-1/message/29669

Regards,
Ernst


David J Taylor GM8ARV 🏴󠁧󠁢󠁳󠁣󠁴󠁿 🇪🇺
 

On 31/05/2021 20:15, Daniele Guardigli wrote:
I tried my configuration with temp folders in laptop ssd drive and final folders in external hdd, all the afternoon. It seems...that works! I'll test it during the next days.
Daniele,

Good to hear of the progress.

Be aware that an SSD has a limited write capacity. Check the total number of terabytes which can be written (TBW) on your SSD and compare it with the expected throughput of the services you have selected.

For BAS, HVS-1 and HVS-2 the throughput approaches 1 TB per day.

Cheers,
David
--
SatSignal Software - Quality software for you
Web: https://www.satsignal.eu
Email: david-taylor@blueyonder.co.uk
Twitter: @gm8arv


David J Taylor GM8ARV 🏴󠁧󠁢󠁳󠁣󠁴󠁿 🇪🇺
 

On 31/05/2021 20:13, Ernst Lobsiger via groups.io wrote:
David,
This SMR crap is rather recent. The WD Blue 1TB 3.5'' is still using CMR recording. I run a couple of those with no problem. On the other hand bigger 2.5'' HDDs are mostly with SMR recording.
The bottom line is: Use CMR for many fast write/read/delete operations, use SMR for write once / read many times (archives, films, impossible for EUMETCast raw data because once the disk is
more or less full data must be deleted and overwritten with the speed of the incomming stream. But SMR cannot cope with that.). This has nothing to do with the OS Windows versus GNU/Linux.
Regards,
Ernst
Ernst,

Thanks for that. I hadn't seen that WD data before.

It seems that I may have been lucky in my choice of HDs in the past!

I do try to keep disks at well under 100% usage, perhaps 50-75% where possible. With desktops or tower PCs, I try to have the EUMETCast disks and "processed data" disks on physically separate spindles (not different partitions, of course!).

Completely agree that SMR should be avoided for EUMETCast applications. Perhaps disks marketed for video surveillance use might be a good starting selection point?

I found this article a useful introduction, but perhaps there's something a little better?


https://www.reichelt.com/magazin/en/smr-cmr-which-hard-drive-is-best-for-which-purpose/

Cheers,
David
--
SatSignal Software - Quality software for you
Web: https://www.satsignal.eu
Email: david-taylor@blueyonder.co.uk
Twitter: @gm8arv


Daniele Guardigli
 

Thanks David,

in order to reduce the amount of data to be stored, is it possible on .ini file to write only services I'm interested in? For example EPS-10, Channel 1,2 and 5?

Il giorno mar 1 giu 2021 alle ore 07:26 David J Taylor GM8ARV 🏴󠁧󠁢󠁳󠁣󠁴󠁿 🇪🇺 via groups.io <david-taylor=blueyonder.co.uk@groups.io> ha scritto:
On 31/05/2021 20:15, Daniele Guardigli wrote:
> I tried my configuration with temp folders in laptop ssd drive and final
> folders in external hdd, all the afternoon. It seems...that works! I'll
> test it during the next days.

Daniele,

Good to hear of the progress.

Be aware that an SSD has a limited write capacity.  Check the total
number of terabytes which can be written (TBW) on your SSD and compare
it with the expected throughput of the services you have selected.

For BAS, HVS-1 and HVS-2 the throughput approaches 1 TB per day.

Cheers,
David
--
SatSignal Software - Quality software for you
Web: https://www.satsignal.eu
Email: david-taylor@...
Twitter: @gm8arv







--
Daniele Guardigli


Ernst Lobsiger
 

On Mon, May 31, 2021 at 10:55 PM, Daniele Guardigli wrote:
Thanks David,
 
in order to reduce the amount of data to be stored, is it possible on .ini file to write only services I'm interested in? For example EPS-10, Channel 1,2 and 5?

 
Daniele,

that's what cast-client-channels_bas.in and cast-client-channels_hvs-1.ini are for. You do not even have to stop your system.
Just comment # out the channels you do not need and beware that you do not have a * entry (all the rest) to a default directory.
These cast-client-channels_*.ini files are reread periodically by the client and reception is adapted accordingly on the fly.

Ernst


David J Taylor GM8ARV 🏴󠁧󠁢󠁳󠁣󠁴󠁿 🇪🇺
 

On 31/05/2021 20:13, Ernst Lobsiger via groups.io wrote:
David,
This SMR crap is rather recent. The WD Blue 1TB 3.5'' is still using CMR recording. I run a couple of those with no problem. On the other hand bigger 2.5'' HDDs are mostly with SMR recording.
The bottom line is: Use CMR for many fast write/read/delete operations, use SMR for write once / read many times (archives, films, impossible for EUMETCast raw data because once the disk is
more or less full data must be deleted and overwritten with the speed of the incomming stream. But SMR cannot cope with that.). This has nothing to do with the OS Windows versus GNU/Linux.
Regards,
Ernst
Interestingly, I note that when I needed to replace a failing 4TB 3.5" disk in my HVS-1/HVS-2 PC I used a video/surveillance disk:

WD40PURZ-85TTDY0

https://documents.westerndigital.com/content/dam/doc-library/en_us/assets/public/western-digital/product/internal-drives/wd-purple-hdd/product-brief-wd-purple-hdd.pdf

Whilst this doesn't appear to state CMR or SMR (at least in the table similar to the one you kindly provided for the WD Blue), it does have:

AllFrame 4K™ Technology

Perhaps it being a surveillance disk is, alone, a statement that it's CMR? I do recall discussing this in the past, so perhaps that's why I went for that disk?

Cheers,
David
--
SatSignal Software - Quality software for you
Web: https://www.satsignal.eu
Email: david-taylor@blueyonder.co.uk
Twitter: @gm8arv