Re: Reducing packet loss

Ernst Lobsiger
 

Dear David,

As you know I use GNU/Linux on ancient hardware with 8GB RAM only.
With these PCs I am able to receive all 3 services with good results.
Having a reasonable reception SNR is not what we are talking about.

On my PCs I had to

1) Disable all kind of C-States in the BIOS
2) Use two HDDs one for OS + locally processed images
   and one for EUMETCast Basic, HVS-1 and HVS-2 (T1 + T2) raw data
3) I used a 3.5GB RAM disk for tmp and received + my script "mvmsg"
   to move received to data HDD and to take care of damaged tmp files
4) I recently gave up the RAM disk taking care not to stress the data
   HDD (now including tmp + received) too much anymore. I do this by:
   - Omitting as far as possible du and find (re WINDOWS your trimtree
     comes to mind and of course things like defrag and virus scanners)
   - Sorting the data in a Channel/YYYY/mm/DD structure and tidy by day
   - Using the big 8000000 socket channel buffers now proposed by EUMETSAT
   - Having now most of the 8GB RAM for HDD raed/write buffering by Linux
5) A further significant step was to pin my busiest interrupts to seperate
   processor cores. For those using routers I propose to pin the NIC
   interrupt and all HDD interrupts to separate cores. It's rather easy
   under Linux (I published a script on this list recently). Maybe someone
   can come up with a similar solution and demonstrate results for WINDOWS.

I always had 2 PC systems though I still locally process all OLCI EFR data of
Sentinel-3A and Sentinel-3B on the receiver. The idea is not to move data
around too much. My receivers NFS export data for processing with PyTroll/SatPy
and EUMETCastView. Other than the receivers my processing PCs do not run 24/7.
An I make no attempt to process e.g. all GOES-16 data to make 4K animations.
I will use 4TB data HDDs in the future to get some more slack. I am convinced
that the two PC system is the future (Thorsten will object :-) route we should
take regarding the amount of data we already get and will soon get with MTGs.


Cheers,
Ernst

Join MSG-1@groups.io to automatically receive all group messages.