Re: Problem with wspr-server ??
toggle quoted messageShow quoted text
The WD clients independently upload spots to two servers:
1) 'Classic spots' (like those from WSJT-x) are sent directly to wsprnet.org
2) 'Enhanced spots' (the same spots with additional information) to wsprdaemon.org
The WD client caches both sets of spots in the non-volatile directory tree ~/wsprdaemon/uploads.d/..... until the destination server acknowledges successfully receiving those spots. WD limits wsprnet.org upload transactions to at most 1000 spots, so after an extended outage it can take minutes or hours for all of the spots in the WD cache to be uploaded to the wsprnet.org server.
There is a diagram of that data flow at: http://wsprdaemon.org/technical.html
Since Nov 20th Jim's jillil.com 'top spotters' site has obtained its spots from the wsprdaemon.org scrape.
The WD server scrapes the wsprnet.org database 3 times during each 2 minute wspr cycle. Occasional timeouts are common, but in the last 24 hours the scraper has been frequently reporting multiple consecutive timeouts. The WD scraper reports missing sequence numbers ('gaps') in those spot reports, but in spite of all the timeouts I see relatively few large gaps in the spot sequence numbers once they are finally delivered.
However, neither the WD client nor the WD scraper can verify if the wsprnet.org database is dropping the spots it acknowledges that it has received. It is on my TODO list to add a 'VALIDATE' option to the WD client's wsprnet.org functions. Until then, I know of nowhere to learn if wsprnet.org has accepted but then invisibly dropped the spots being uploaded. I can think of no way for the WD scraper to more fully validate the spot data it receives from its API call to the wsprnet.org server.
On Sun, Dec 20, 2020 at 9:22 AM ON5KQ <ON5KQ@...> wrote: