What happens with old spots not yet uploaded (probably because of Internet failure) but stored in txt files...


ON5KQ
 

Dear,
During last night I probably had an Internet connection failuer, so from 20h utc yesterday until 4h utc in the morning, there was nothing uploaded.
The wsprdaemon script created a lot of txt files instead (containing the decoded spots during that time). In the meantime the wsprdaemon.conf file switched from nighttime to daylight schedule and a little later during the new daylight schedule the Internet came back.

Because of the new daylight schedule running, the wsprdaemon script didn't realize , that from the nighttime schedule (not active anymore) there were still a lot of ,txt files containing the spots to be uploaded. 
If I remember correctly all the .txt files with the nighttime spots were in this directory
/home/wsprdaemon/uploads.d/wsprnet.d/spots.d/ ....   and channel by channel the txt files were written for every 2min of transmission.

I was wondering, why the waiting spots were not uploaded immediately as the Internet came back in the morning...

To find out,  I restarted with "./wsprdaemon.sh -vvva"  to force high verbosity to check the various log files.
I found that the script searched for txt files to be uploaded, but not in the nighttime folder, but the current job folder for the running daytime schedule - so it would not immediately find the hundreds of waiting txt files only after roughly another 12h, when the night schedule becomes active again.

However, I didn't want to wait that long so...
(I am not sure, if it was correct, what I did then....):
- to upload the waiting txt files immediately, I stopped wsprdaemon with
./wsprdaemon.sh -z
then changed wsprdaemon.conf in that way, that the nighttime schedule was resumed instead of starting the daytime schedule.
Started again with:
./wsprdaemon.sh -vvva

Now In the logs I could see, that immediately more than 300 txt files waiting for upload were found from the script
Quickly the logs also showed that after a while many many uploads were done - it must be thousands of spots...

After a while, all txt files were erased and I found in the curl-logs the confirmation of upload acceptance... again thousands. I was wondering however that all that was only a matter of minutes - the missing upload was very quick...

So, I was happy and stopped wsprdaemon, as I did not want to write too much data in the log-files with that high verbosity level.
So, I restarted with
./wsprdaemon.sh -a

Now what::
Hours later now, still there are no spots at all between 22h yesterday and 6h this morning - neither on wsprnet.org, nor on vk7jj's website...

It would be great to understand what really happened with the uploaded spots from partly today and partly from yesterday (based on utc time)

Will the missing spots ever appear ?
Why I got a confirmation of acceptance for thousands of spots in the curl-file, when these spots never appear ?
What did I wrong, that still no missing spot is delivered and how should I proceed next time to prevent data loss.

Or do just need to wait longer for the data to appear (may be one day or so ?!)
I am confused...hi

May be Rob can explain in the next zoom-meeting, how the upload of old spots really works - either because of Internet connection problems or even after portable kiwi operation in the field with later upload from home (Glenn's application ?!)

regards,
Ulli, ON5KQ

Join wsprdaemon@groups.io to automatically receive all group messages.