RamDisk size for HVS-2


Alan Curnow
 

Hello,

I recently installed a second tuner (TBS5927) for HVS-2 (Sentinal OLCI and SLSSTR) to work alongside my SR1 for Basic and HVS-1.

My 500MB ramdisk was fine for Basic and HVS-1 tmp files using only around 50MB of it. I increased it to 4GB but it's still too small for HVS-2 tmp files. I've had to move HVS-2 tmp to an SSD which isn't ideal and it's currently using 4.76GB in 21 files. It seems to need several roughly 15 min bursts of S3A and then S3B data with long pauses between bursts before the tmp files are completed and get automatically deleted.

I can increase the Ramdisk to about 5.5GB but that may not be sufficient. What size Ramdisk does anyone else use for HVS-2 which works OK or am I safer using the SSD.

Cheers,
Alan


Ernst Lobsiger
 

On Thu, Oct 14, 2021 at 09:31 AM, Alan Curnow wrote:
I can increase the Ramdisk to about 5.5GB but that may not be sufficient. What size Ramdisk does anyone else use for HVS-2 which works OK or am I safer using the SSD.
Alan,

this is an old question I wrote a couple of papers about. The short answer is:

I have several receivers that take BAS + HVS-1 + HVS-2 on one single 3GB RAM-Disk.

I never had a Windows receiver but this can be done on either GNU/Linux or Windows.
One of the main problem is that broken tmp files can stay for hours waiting for
retransmissions that never arrive. So if it seems you have a solution for good
weather the first rain can produce missed packets and then fill up your RAM disk.
I have my own solutions for that but probably nobody uses my Windows MVMSG.cmd.

I asked EUMETSAT long ago to add a CLI switch to the TelliCast client to instantly
delete broken tmp files. This will make handling minimal RAM disks much more easy.
Of course EUMETSAT had to discuss that with NEWTEC that holds all the source code.
According to OPS the next client will now have this (my) switch. Don't hold your breath.


Regards,
Ernst


Alan Curnow
 

Thanks Ernst for your detailed response.

I've been monitoring my HVS-2 temp files for the last six hours. When first checked there was just one 20MB file there even though it had been working for 12 hours up to that point. Over the next two hours it increased to a maximum of 21 files totalling 4.9GB. It stayed around that figure for an hour or so and then started decreasing in the number of files and memory used to a minimum of 4 files and 850MB after around 5 hours. It's now slowly increasing again.

I notice that none of the existing temp files last update status are more than 2 hours old so it seems all are being written to, and not broken, unless Tellicast is actually deleting them if they haven't been accessed in a certain time. I have a 1.8m dish in SW England and the SR1 reports a Es/No of 17.6dB. In heavy rain it has dropped to 16.9dB. The TBS5927 reports a S/N of 16.4dB and a BER of 0.000000, signal quality of 82 using IPTool v3.0.5.0

It's odd that on your system you never exceed 1.5GB used on the Ramdisk, unless the Windows and Linux versions of Tellicast are different in some way.

Checking the hvs-2 log it does report a missed/incomplete file around avery 10 mins on average.

Cheers,
Alan


Ernst Lobsiger
 

<cite>
It's odd that on your system you never exceed 1.5GB used on the Ramdisk, unless the Windows and Linux versions of Tellicast are different in some way.
</cite>

Alan,

given your 1.8m dish and the experiments you just described you are certainly not an average Windows TelliCast user. I also had a look at your rare posts that show that you have the potential to understand and use my MVMSG.cmd. You still find the DISTRO 0.22 in the MSG-1 file section (login, go to files and search for MVMSG). The zip contains my papers with the technical background. I have written this on my Windows 7 desktop before T2 and HVS-2 even existed. It uses an NTFS-formatted RAM-disk. The reason for this are rather technical except for the fact that I had no Windows RAM-disk at all and I found it outright ridiculous to have to pay for such an ADD-on. I actually simulated a Windows 7 TelliCast receiver (something I never had and never will have) and a RAM-disk with an external ZIP-100 drive that I formatted FAT32 and NTFS.

David uses and describes a much simpler approach based on robocopy.exe and trimtree.exe on his site but I doubt this can do what my MVMSG.cmd actually does. I also note that he uses RAM-disk sizes up to 8GB (see e.g. his PC Lund).

https://www.satsignal.eu/wxsat/Ayecka/RAMdisk-HVS.html

https://www.satsignal.eu/mrtg/lund-ramdisk-trees.html

The only persons I know to have used MVMSG.cmd are well known Arne van Belle (my Beta tester for this project) and Emmanuel "Manu" Dupré (he made me brush up my frech). I also made a special version that can handle 4 TC clients. Arne used this version to demonstrate that his TBS-6983 could receive two full EUMETCast transponders by setting up a system that received 2 x T1 = 2 x (BAS + HVS-1). Before that mighty demonstration (though I always knew) it was very much under dispute and even FUD whether this novel PCIe card from a "Chinese startup" could ever receive both T1 and future T2. I just found that version MVMSG4.cmd as configured by Arne van Belle in an old e-mail. FYI I attach it to this post. MVMSG4.cmd could be used for BAS + HVS-1 + HVS-2 + future HVS-3. And you might be the person that makes it fly. AFAIK there is almost no difference between GNU/Linux and Windows versions of the TC client except for the fact that (impossible under GNU/Linux) Windows can put tmp and received files on different filesystems (partitions) what YOU SHOULD NEVER DO in the first place. Actually I see no reason why you could not run all three Windows clients on a 3-4GB RAM-disk (though I cannot test it here).

Cheers,
Ernst


P.S.
I just tested MVMSG.cmd on my Windows 10 PRO 64Bit desktop. In demo mode it still works as under Windows 7. If you really want to go for it read my documentation first and then send me a PM describing your system. I'll see what I can do.


David J Taylor GM8ARV 🏴󠁧󠁢󠁳󠁣󠁴󠁿 🇪🇺
 

On 15/10/2021 09:18, Ernst Lobsiger via groups.io wrote:
David uses and describes a much simpler approach based on robocopy.exe and
trimtree.exe on his site but I doubt this can do what my MVMSG.cmd actually
does. I also note that he uses RAM-disk sizes up to 8GB (see e.g. his PC Lund).
Here, at the extremities of the satellite footprint, and with a dish that is
too sensitive to wind, I have much worse reception conditions than those in
central Europe!

The simple Robocopy approach works extremely well, and is much easier to
understand.

My PC Lund receives the full HVS-1 and HVS-2 on a single card:

https://www.satsignal.eu/mrtg/lund-ramdisk-trees.html

My PC Kiruna receives HVS-1 along and has no RAMdisk. Data is sent directly to
a separate 1 TB CRM hard disk.

Yes, Lund has an 8 GB RAMdisk, but you might argue that if there are such
issues that a RAMdisk that big is needed, will the data be good enough to
justify reception in the first place?

Recent experience is that when the GOES-16/17 data goes offline from the USA,
and EUMETSAT attempt to catch up, the HVS-1 data stream is overloaded in any
case. I see this both here and at Arne's.

The guidance? Get a 32 GB PC and be prepared to devote 8 GB to a RAMdisk.
Monitor it, and trim downwards as meets your particular needs.

Cheers,
David
--
SatSignal Software - Quality software for you
Web: https://www.satsignal.eu
Email: david-taylor@blueyonder.co.uk
Twitter: @gm8arv


Ernst Lobsiger
 

On Fri, Oct 15, 2021 at 08:17 AM, David J Taylor GM8ARV 🏴󠁧󠁢󠁳󠁣󠁴󠁿 🇪🇺 wrote:
The simple Robocopy approach works extremely well, and is much easier to
understand.
David,

I agree that MVMSG.cmd is hard to understand which is not my fault as Windows CMD is rather a botch than a shell. One of the big problems was to get file modification times and actual file sizes while the temp files are open and the client is writing. NTFS was just lying and FAT32 had other problems I don't remember. I also agree with your trim down guidance as I have one of my Fujitsu/Siemens 2009 Core2 Duo with 8GB RAM, 2 old Hitachi CMR disks and no RAM disk that can run at 100% system load receiveing with a 20 Euro Skystar 2 eXpress HD catching up with LaPalma RSS frames from some days back and producing webm videos with ffmpeg at the same time without loosing a single multicast packet :-).

Cheers,
Ernst


Alan Curnow
 

Thanks again Ernst for your information. I'll read the info you posted with interest and report back. Also thanks to David for his info.

This afternoon I checked and the swap file was 1 file of 100MB and 2mins old, so all the temp files that were created yesterday must have been genuine and correct and the 4.9GB peak I reached yesterday was all genuine valid data so it seems couldn't have been cleared earlier to free space.
Interestingly for the past few hours the hvs-2 swap usage has only peaked at around 500MB, presumably as only one Sentinal satellite happens to be downloading at a time.

For Bas and HVS-1 I do write the Tellicast received files to the ramdisk too, and wrote a Windows program to transfer them to two directory structures on my 2TB EUMETCast storage disk every 30 secs, one a copy of the received directories for use by Satpy or EumetCastView, while the other directory structure is the one needed by Satsignal software with some directories renamed or combined. This is so Satsignal can delete them when processed. I did try not deleting them so all programs could use the one directory structure but Satsignal processing took much longer as it kept searching through thousands of files for new ones. This only used 100MB of ramdisk in total so was no problem.

This won't work for HVS-2 as completed files are so much larger, so are written direct to the main EUMETCast storage disk. For the moment I may try risking using a 5.5GB ramdisk for hvs-2 tmp files too, and see how long it survives for. :)

I'm only just starting trying out Satpy/trollImage and Python so am not very good at it yet. I'm used to C and C++, so hopefully will get the hang of it quickly.

Cheers,
Alan


Ernst Lobsiger
 

Alan,

if you have already solved the transfer problem of received RAM-disk --> HDD then remains the broken tmp files. In one post you said to have a missed/incomplete HVS-2 files every 10 minutes. This is far too high for whatever reason (with your SNR of 16dB you should rather have one missed/incomplete file per day). Assuming these are OLCI EFR files of typically 700MB would mean you get 700MB more every 10 minutes that hang around for the next 3 hours.

This is in a nutshell what I found out: EUMETSAT can set a kind of timeout per product disseminated. This used to be 3 hours for OLCI EFR files but can be up to 24 hours for other low priority stuff according to KP. When you have missed (missed/unrecovered = lost) packets in the stream to that file it will hang around as incomplete tmp (actually full 700MB but with some 0 filled hole in it) and wait to be fixed. After the set timeout the client will delete a still incomplete tmp. It can be fixed by retransmissions within the set delay that EIP (Extremly Important Persons) can ask for (NACKs). Perhaps EUMETSAT also has a network of reference receivers so that they can detect widespread reception problems (maybe caused by their uplink) and initiate retransmission for those specific cases automatically. But the fact is most missed/incomplete files we get are caused by our local receiver problems EUMETSAT will never know about and will certainly not send some more extra packets for an amateur user in Cyprus or Sweden. That's why I asked EUMETSAT for the client switch I was talking about. With that switch every station that wants to use RAM-disks can decide whether the client should delete missed/incomplete files immediately. In this case you can forget tmp file management and just organize the RAM-disk ---> HDD transfer for fully received files. As a matter of fact, as the Windows client can even have tmp and received on different file systems (I don't recommend that), you could just have an unmanaged RAM-disk as a dynamic buffer for tmp files. More details in my two two papers inside the MVMSG 0.22 distribution that you will certainly understand.


Regards,
Ernst

P.S.
Just recently we suffered from sun satellite antenna colinearity (solar outage) that occurs twice a year.
This of course is a main RAM-disk filler event if broken HVS-2 tmp files are not deleted fast enough.


Thorsten Miglus
 

Hello,

The removal of OLCI Level-1 Full Resolution Near Real Time data from EUMETCast Satellite will take place on 15 December 2021.

Cheers,
Thorsten


Alan Curnow
 

Ernst,

From your last post it would appear that all those tmp files 1 to 2 hours since last modified are actually broken and cluttering up the ramdisk. On David's PC Lund graphs his tmp files only exceeded 5MB twice in a year and only once over 5.5MB so I may get away with my Ramdisk as it is though it would be nice not to have such clutter.

On your documentation for MVMSG you also stored your hvs-2 received files on the ramdisk before moving them. As they can be 700MB or so in size I didn't think that was feasible but it seems to work fine for you in 4GB as long as you get rid of the broken tmp files. I'll send you a PM of my setup as I'd be interested in doing that.

Regarding log errors I exaggerated the 10 mins, over the last 24 hours it was

HVS-2
17 entries for 'Missed parts of file....'
22 entries for 'Lost  xx messages in the last 60 seconds...' right after disconnecting from a channel. Don't know if that is a problem.

HVS-1
Hundreds of file missing/incomplete entries referring to GOES 17. No errors apparantly on other satellites.

Basic
No Lost or Missed entries.
Hundreds of file missing/incomplete messages right after disconnecting from a Data Channel 1, 2 and 5 channels like this
MSG:2021-10-15 00:49:07.818:File transmission eefa8c14031f25ad interrupted: 6 files missing/incomplete 
Not sure how critical they are as my Satsignal checks of MSG, Himawari, Metop etc images always seems complete without missing or corrupt segments.

Cheers,
Alan
 


David J Taylor GM8ARV 🏴󠁧󠁢󠁳󠁣󠁴󠁿 🇪🇺
 

The occasional errors in dissemination appear to have remained in the last day
or two, so caution may be needed in interpreting changes!

I handle the stale "tmp" files with a simple TrimTree command:

TrimTree 120m Z:\EUMETCast\tmp\ *

which is run regularly.

There are changes afoot for Metop-A and Sentinel data over EUMETCast over the
next couple of months, but I don't yet know what is planned to fill the
bandwidth vacated! GOES-16/17 data has been subject to interruption.

In the last couple of days I noticed 250m and 500m wind data incoming.
Although this was caught by a daily trim of \received\, I added it to a regular
TrimTree of data which I don't use.

"W_NL-KNMI-DeBilt,SURFACE+SATELLITE,HY-2?+HSCAT_C_EHDB_2*_o_*_ovw_l2.bin"

Cheers,
David
--
SatSignal Software - Quality software for you
Web: https://www.satsignal.eu
Email: david-taylor@blueyonder.co.uk
Twitter: @gm8arv


Ernst Lobsiger
 

Alan,

first of all, if you have not done that already, you should use a TC log analizer like TClogSummary.cmd for each client. It's more than 10 years back (old DVB-S client) I wrote the first version with ideas from Arne van Belle. On MSG-1 in the file section you will find TClogSummar091.zip. That's a slightly deprecated distribution but has all explanations you might need. What you should actually use is my latest version 0.95 (at least that's the latest I can find here in my Windows 10 disorder) of the script that I attach to this e-mail. I also attach the output of TClogSummary 0.95 from yesterday of my GNU/Linux receiver Io. Arne and David have used TClogSummary from the beginning and include the output in their reports to EUMETSAT. Under GNU/Linux I use a BASH version of course.

There is two points I'd like to clarify (as a short cut to reading my two pdf reports in the MVMSG DISTRIBUTION 022):

1.) If you have tmp on a RAM-disk and then also add received on the same RAM-disk this does not change anything in terms of RAM-disk size used to start with. When fully received tmp files are moved (which on the same filesystem means just renamed by hard links) they disappear as tmp files and reappear like magic as received files in their respective channel directories (no data is moved or added). Of course now it's your turn do move them to your big HDD or SSD (where move now translates to copy and delete).

2.) There are to kinds of closed tmp files that can hang around and fill your RAM-disk.
a) Broken tmp files that are waiting to be fixed. If not fixed deleted by the client after timeout set by EUMETSAT.
These usually go with
... File transmission ... ended: filelist missing/incomplete
... File transmission ... ended: N file(s) missing/incomplete
... Missed parts of file
... Missed file
messages in the log (counted and filtered out by TClogSummary, see also TD15).

b) Incomplete tmp files from transmissions that are interrupted (by EUMETSAT uplink) for priority reasons. There is nothing wrong with this kind of tmps as transmission will be resumed later and the respective files finally received.
These usually go with
... File transmission ... interrupted: filelist missing/incomplete
... File transmission ... interrupted: N file(s) missing/incomplete
messages in the log (only counted by TClogSummary, see also TD15).

The real problem is that neither MVMSG nor the newer solution David uses can differentiate between a) and b). If you just delete closed tmps that have not been modified for some set delay there is always a chance that you also delete a file of kind b). The longer you set the delay the chance of deleting type b) goes down but the size that your RAM-disk must provide goes up. With my proposed switch for the TC client we will only get active tmps and harmless kind b) waiting tmps.


I usually delete inactive tmps on RAM-disks after 25 minutes for BAS and after 5 minutes for HVS-1 and HVS-2.


Regards,
Ernst

P.S.
I will reply to your PM. Meanwhile you must/should get TClogSummary.cmd running for all your three clients.


Alan Curnow
 

Thanks Ernst & David,

I have David's Trimtree running on the 2T EUMETCast data disk to keep from filling the disk, but never needed it on bas and hvs-1 tmp files as I never had a problem with them building up. Running it on just hvs-2 tmp may be all that's needed to keep them under control though I'll see how Ernst's MVMSG does as he's kind enough to help with it.

Your description of the tmp file behaviour makes sense Ernst. I didn't know whether the final received file was constructed from the tmp files which were then deleted or just renamed when complete.

Good to know that the 'interrupted' messages are not a receiving problem.

I'll get TClogSummary installed as it's very useful. I was thinking of writing a Windows program to do a similar result. May still do so if I'm bored. :)

Cheers,
Alan


Ernst Lobsiger
 

On Sat, Oct 16, 2021 at 09:44 AM, Alan Curnow wrote:
Thanks Ernst & David,

I have David's Trimtree running on the 2T EUMETCast data disk to keep from filling the disk, but never needed it on bas and hvs-1 tmp files as I never had a problem with them building up. Running it on just hvs-2 tmp may be all that's needed to keep them under control though I'll see how Ernst's MVMSG does as he's kind enough to help with it.
Alan and David,

I made a couple of tests and discovered that the behaviour of CMD and NTFS has slightly changed from Windows 7 32Bit to Windows 10 64Bit. The problem is getting metadata like timestamps and size in bytes of an open tmp file. Under Windows 7 it was possible with a CMD trick to force NTFS to return current metadata values. So I checked the metadata updated and checked again. This way I could detect the file size growing. Windows 10 ignores this trick and only updates meta data from time to time after some amount of data has been written. This makes MVMSG.cmd obsolete as I used this metadata magic to prevent the script as much as possible from trying to move open tmp files.

Using trimtree and last modification times to manage these life tmp folders as David does will probably try to delete open files and the folders they live in then and when. In these cases as a last stop the OS will just say "access denied". Copying received data with robocopy is certainly a good thing. I have used it to copy folders as well. MVMSG did both management and copy while David's solution has two parts.

@Alan as you already have and use trimtree I suggest you use it to manage the tmp folders the way David proposes. You should also manage HVS-1 and BAS tmp folders. For received transfer you update your program to include HVS-2.

@David I propose a minor update of your description "Using a RAMdisk for HVS-1 and HVS-2" on www.satsignal.eu

<CITE>
REM Script scheduled to run every ten minutes

TrimTree 15m Z:\EUMETCast\tmp\bas *
TrimTree 6m Z:\EUMETCast\tmp\hvs-1 *
TrimTree 6m Z:\EUMETCast\tmp\hvs-2 *
<CITE>

Scheduled every 10 minutes means that a tmp can get 25m (BAS), 16m (HVS-1) and 16m (HVS-2) old since last write.
I schedule mvmsg every 2 minutes and delete HVS tmps not modified for 6m. So my maximum tmp age is 8m unmodified.
The 16m for HVS is certainly safe but asks for very BIG RAM-disks e.g. in case of broken Sentinel-3X 700MB files.


Cheers,
Ernst


Alan Curnow
 

Ernst,

Thanks for your efforts in analysing Windows 10 behaviour and finding MSMVG now has no real advantage over Trimtree. I'll do as you suggest and update my program to moving received files to include hvs-2.

When my hvs-2 tmp files reached 4.9GB there were many files over 1 hour old and around 2 that were 2 hours old. I could probably use a Trimtree time of 15min or so and still save a lot of space, which may be safer than 6m in not deleting possibly valid tmp files.

It would be better though to determine why I'm losing so many hvs-2 files causing broken tmps while you aren't.

Here's my TClogSummaries for the same period as you. Bas is fine, and hvs-1 is similar, though misses 22 more GOES files than you did, while hvs-2 has 18 missed Sentinel files while you had none.

I'm using iptool with the TBS5927 for hvs-2. Several people reported problems using iptool, and BDADataEx seemed to be the preferred program. Do you think it's worth me trying BDADataEx instead, or do you think that's not a likely reason.

Cheers,
Alan


Ian Deans
 

On 16/10/2021 23:11, Alan Curnow wrote:
Here are my TClogSummaries from yesterday. Bas looks good, and hvs-1 has bad GOES files though 22 more than you had Ernst, while hvs-2 missed 18 files probably accounting for the excess broken tmp files, while you had none. Most perplexing.
I'm using iptool on the tbs5927 for hvs-2. There seem to be issues with iptool reported by some people. Is it worth trying BDADataEx instead which seems to be the preferred choice of many or do you think that's unlikely to be the cause.
Cheer,
Alan
==============================================================================

Alan I suspect your issue is likely to be the TBS 5927.

I have 3 receivers SR1, TBS 5925 and 5927 and the SR1 and 5925 run 14 hours a day without any issues or losses. The 5927 gives me losses even if just used for the basic service.

I put a mail into the group over 6 months ago asking if anyone was using a TBS 5927 without losses and got no replies. I am sure it is a driver problem, but nevertheless it is a nuisance as the 5927 is probably the only one of my 3 receivers I could now replace.

Regards
Ian.


Alan Curnow
 

Ian, 

Thanks for the information. That's a pity though, as you say it seems to be the only one available now unless you go for a PCIe card.

Regards,
Alan


David J Taylor GM8ARV 🏴󠁧󠁢󠁳󠁣󠁴󠁿 🇪🇺
 

On 16/10/2021 23:04, Ernst Lobsiger via groups.io wrote:
Using trimtree and last modification times to manage these life tmp folders as
David does will probably try to delete open files and the folders they live in
then and when. In these cases as a last stop the OS will just say "access
denied". Copying received data with robocopy is certainly a good thing. I have
used it to copy folders as well. MVMSG did both management and copy while
David's solution has two parts.
The program I use simply issues the "delete" command, and if the OS says
"Sorry, Dave. I can't do that..." the file is left untouched. Same with
non-empty directories. I recall there was talk of tmp files being closed and
re-opened under some circumstances, but in practice this hasn't been a problem.

My solution has two parts because there is no need to use the RAMdisk for
\received\ as well as for \tmp\. It gives the user the choice.

Cheers,
David
--
SatSignal Software - Quality software for you
Web: https://www.satsignal.eu
Email: david-taylor@blueyonder.co.uk
Twitter: @gm8arv


David J Taylor GM8ARV 🏴󠁧󠁢󠁳󠁣󠁴󠁿 🇪🇺
 

On 16/10/2021 23:04, Ernst Lobsiger via groups.io wrote:
@David I propose a minor update of your description "Using a RAMdisk for HVS-1
and HVS-2" on www.satsignal.eu

<CITE>
REM Script scheduled to run every ten minutes

TrimTree 15m Z:\EUMETCast\tmp\bas *
TrimTree 6m Z:\EUMETCast\tmp\hvs-1 *
TrimTree 6m Z:\EUMETCast\tmp\hvs-2 *
<CITE>

Scheduled every 10 minutes means that a tmp can get 25m (BAS), 16m (HVS-1) and
16m (HVS-2) old since last write.
I schedule mvmsg every 2 minutes and delete HVS tmps not modified for 6m. So my
maximum tmp age is 8m unmodified.
The 16m for HVS is certainly safe but asks for very BIG RAM-disks e.g. in case
of broken Sentinel-3X 700MB files.


Cheers,
Ernst
Thanks, Ernst. I've added your note below the script. It's good to allow
everyone to tune things easily to meet their own needs, and to appreciate the
background behind the settings!

Just checked and for HVS-2 I run every 4 minutes with a TrimTree setting of 3
minutes!

Cheers,
David
--
SatSignal Software - Quality software for you
Web: https://www.satsignal.eu
Email: david-taylor@blueyonder.co.uk
Twitter: @gm8arv


Ernst Lobsiger
 

On Sun, Oct 17, 2021 at 12:11 AM, David J Taylor GM8ARV 🏴󠁧󠁢󠁳󠁣󠁴󠁿 🇪🇺 wrote:
I recall there was talk of tmp files being closed and
re-opened under some circumstances, but in practice this hasn't been a problem.
David,

Thats the problem of "transmission interrupted" (for priority reasons). These tmps are closed and reopened when transmission is resumed. Some of those are closed and reopened many times. I have an example in my MVMSG docs:

MSG:2016-04-30 11:37:25.964:File transmission 5216cdf900b58902 interrupted: 3 files missing/incomplete
MSG:2016-04-30 11:38:45.266:File transmission 5216cdf900b58902 interrupted: 3 files missing/incomplete
MSG:2016-04-30 11:38:59.007:File transmission 5216cdf900b58902 interrupted: 3 files missing/incomplete
MSG:2016-04-30 11:39:09.251:File transmission 5216cdf900b58902 interrupted: 3 files missing/incomplete
MSG:2016-04-30 11:40:14.874:File transmission 5216cdf900b58902 interrupted: 3 files missing/incomplete
MSG:2016-04-30 11:40:28.801:File transmission 5216cdf900b58902 interrupted: 3 files missing/incomplete
MSG:2016-04-30 11:40:44.665:File transmission 5216cdf900b58902 interrupted: 2 files missing/incomplete
MSG:2016-04-30 11:41:14.960:File transmission 5216cdf900b58902 interrupted: 2 files missing/incomplete ….
This sequence has been extracted by grep from the BAS client log (messages in between not shown) …. ….
many more interruptions, as you can see the whole transmission lasts from 11:37 to 11:53 for 3 files …..
MSG:2016-04-30 11:52:00.424:File transmission 5216cdf900b58902 interrupted: 2 files missing/incomplete
MSG:2016-04-30 11:52:13.970:File transmission 5216cdf900b58902 interrupted: 2 files missing/incomplete
MSG:2016-04-30 11:52:30.633:File transmission 5216cdf900b58902 interrupted: 1 file missing/incomplete
MSG:2016-04-30 11:52:44.153:File transmission 5216cdf900b58902 interrupted: 1 file missing/incomplete
MSG:2016-04-30 11:52:59.834:File transmission 5216cdf900b58902 interrupted: 1 file missing/incomplete

That's a problem if we delete those while closed. Its what I described for Alan further down as type b).
Maybe we just don't take (are not interested in) that low priority stuff and so it's no real problem??!!

Cheers,
Ernst