Re: Problem with wspr-server ??


Rob Robinett
 

The WD clients independently upload spots to two servers:

1) 'Classic spots' (like those from WSJT-x) are sent directly to wsprnet.org
2) 'Enhanced spots'  (the same spots with additional information) to wsprdaemon.org

The WD client caches both sets of spots in the non-volatile directory tree ~/wsprdaemon/uploads.d/..... until the destination server acknowledges successfully receiving those spots.  WD limits wsprnet.org upload transactions to at most 1000 spots, so after an extended outage it can take minutes or hours for all of the spots in the WD cache to be uploaded to the wsprnet.org server.

There is a diagram of that data flow at: http://wsprdaemon.org/technical.html

Since Nov 20th Jim's jillil.com 'top spotters' site has obtained its spots from the wsprdaemon.org scrape.
Phil's wsprd.vk7jj.com site and Peter's WsprWatch IOS app also use the wsprdaemon.org scrape.

++++++++++

The WD server scrapes the wsprnet.org database 3 times during each 2 minute wspr cycle. Occasional timeouts are common, but in the last 24 hours the scraper has been frequently reporting multiple consecutive timeouts.  The WD scraper reports missing sequence numbers ('gaps') in those spot reports, but in spite of all the timeouts I see relatively few large gaps in the spot sequence numbers once they are finally delivered.

However, neither the WD client nor the WD scraper can verify if the wsprnet.org database is dropping the spots it acknowledges that it has received.  It is on my TODO list to add a 'VALIDATE' option to the WD client's wsprnet.org functions.  Until then, I know of nowhere to learn if wsprnet.org has accepted but then invisibly dropped the spots being uploaded.  I can think of no way for the WD scraper to more fully validate the spot data it receives from its API call to the wsprnet.org server.

Rob

On Sun, Dec 20, 2020 at 9:22 AM ON5KQ <ON5KQ@...> wrote:
Hi Gwyn,
Does WA2ZKD's list reflect the spots in the wsprnet.org site, or does it show wsprdaemon database ?

New spots are showing up now again on the WA2ZKD site as well as on Phil's website wsprd.vk7jj

However where are the older spots from today - only 9000 spots showed up so far - it should be about 20000 instead at this time... Are these spots stored somewhere or lost ?
The reason,  why I want to know is, that I do not know, what to monitor is such a case as today. Perhaps I should have taken action, as there is something wrong here with my PC ?
However:
- I know, decoder worked ok
- I know in the afternoon all decodes where stored in many files waiting to be accepted as uploaded. However it doesn't happen. Instead at a certain point all files with old spots from today disappeared and never showed up again... at least not on WA2ZLD site as well as VK7JJ's site.

There is a gap with no spots from 8h28utc to 16h46 utc. with no spots in between, which explains only 9000 spots so far... Does it mean both websites are not showing wsprdeamon database but wsprnet.org, which is faulty ?
Unfortunately I do not understand the data flow.... hope to learn it sometimes...hi

73s
Ulli, ON5KQ



--
Rob Robinett
AI6VN
mobile: +1 650 218 8896

Join wsprdaemon@groups.io to automatically receive all group messages.