Date
1 - 10 of 10
Problem with wspr-server ??
Stu C
Might be a loading/DDOS thing as I can see some of my spots (one band) but tried to refresh for all bands and no luck.
The aggregation site pe1itr.com failed to tally today.
On WA2ZKD's site i'm too far up the listing today so might be worse from some locations as a lot of the top users are missing spots?
Stu
The aggregation site pe1itr.com failed to tally today.
On WA2ZKD's site i'm too far up the listing today so might be worse from some locations as a lot of the top users are missing spots?
Stu
ON5KQ
13h50 utc
can't find any spots on the wsprdaemon server database when connecting via Grafana... since early morning all bands.
Originally I saw a lot of files on my local workstation in the wsprdaemon folder, waiting to be uploaded, however they are all erased now.
Curl.log only zeros... no upload confirmations...
There must be something broken... in the meantime, I keep all running here, although it doesn't seem that decodes are written in files, because there is a network error....
The merge.log shows all the spots from the kiwi on each band, but nothing arrived at the wsprdaemon server...
Ulli, ON5KQ
can't find any spots on the wsprdaemon server database when connecting via Grafana... since early morning all bands.
Originally I saw a lot of files on my local workstation in the wsprdaemon folder, waiting to be uploaded, however they are all erased now.
Curl.log only zeros... no upload confirmations...
There must be something broken... in the meantime, I keep all running here, although it doesn't seem that decodes are written in files, because there is a network error....
The merge.log shows all the spots from the kiwi on each band, but nothing arrived at the wsprdaemon server...
Ulli, ON5KQ
Gwyn Griffiths
I've emailed Rob with a note that many of the API requests to wsprnet.org are timing out and there are large gaps in the sequence numbers of spots being returned by wsprnet.org. There is certainly congestion at wsprnet,org today.
However, I've found that upload into the wsprdaemon_spots table in the tutorial database is working well, as this is entirely independent of wsprnet.org. I know terminology can sometimes be confusing, so in case it helps, here is a reminder:
On any wsprdaemon server (wsprdaemon.org, logs1.wsprdaemon.org, logs2.wsprdaemon.org):
1. Database 'wsprnet' has a table 'spots' gathered via an API from wsprnet.org
2. Database 'tutorial' has two tables 'wsprdaemon_spots' and 'wsprdaemon_noise' from data sent directly to the wsprdaemon server from your client wsprdaemon application, and so is unaffected by any congestion at wsprnet.org
regards
Gwyn G3ZIL
However, I've found that upload into the wsprdaemon_spots table in the tutorial database is working well, as this is entirely independent of wsprnet.org. I know terminology can sometimes be confusing, so in case it helps, here is a reminder:
On any wsprdaemon server (wsprdaemon.org, logs1.wsprdaemon.org, logs2.wsprdaemon.org):
1. Database 'wsprnet' has a table 'spots' gathered via an API from wsprnet.org
2. Database 'tutorial' has two tables 'wsprdaemon_spots' and 'wsprdaemon_noise' from data sent directly to the wsprdaemon server from your client wsprdaemon application, and so is unaffected by any congestion at wsprnet.org
regards
Gwyn G3ZIL
ON5KQ
Hi Gwyn,
Does WA2ZKD's list reflect the spots in the wsprnet.org site, or does it show wsprdaemon database ?
New spots are showing up now again on the WA2ZKD site as well as on Phil's website wsprd.vk7jj
However where are the older spots from today - only 9000 spots showed up so far - it should be about 20000 instead at this time... Are these spots stored somewhere or lost ?
The reason, why I want to know is, that I do not know, what to monitor is such a case as today. Perhaps I should have taken action, as there is something wrong here with my PC ?
However:
- I know, decoder worked ok
- I know in the afternoon all decodes where stored in many files waiting to be accepted as uploaded. However it doesn't happen. Instead at a certain point all files with old spots from today disappeared and never showed up again... at least not on WA2ZLD site as well as VK7JJ's site.
There is a gap with no spots from 8h28utc to 16h46 utc. with no spots in between, which explains only 9000 spots so far... Does it mean both websites are not showing wsprdeamon database but wsprnet.org, which is faulty ?
Unfortunately I do not understand the data flow.... hope to learn it sometimes...hi
73s
Ulli, ON5KQ
Does WA2ZKD's list reflect the spots in the wsprnet.org site, or does it show wsprdaemon database ?
New spots are showing up now again on the WA2ZKD site as well as on Phil's website wsprd.vk7jj
However where are the older spots from today - only 9000 spots showed up so far - it should be about 20000 instead at this time... Are these spots stored somewhere or lost ?
The reason, why I want to know is, that I do not know, what to monitor is such a case as today. Perhaps I should have taken action, as there is something wrong here with my PC ?
However:
- I know, decoder worked ok
- I know in the afternoon all decodes where stored in many files waiting to be accepted as uploaded. However it doesn't happen. Instead at a certain point all files with old spots from today disappeared and never showed up again... at least not on WA2ZLD site as well as VK7JJ's site.
There is a gap with no spots from 8h28utc to 16h46 utc. with no spots in between, which explains only 9000 spots so far... Does it mean both websites are not showing wsprdeamon database but wsprnet.org, which is faulty ?
Unfortunately I do not understand the data flow.... hope to learn it sometimes...hi
73s
Ulli, ON5KQ
Gwyn Griffiths
Hello Ulli
Some time ago I put a simplified data flow at the bottom of the webpage at: http://wsprdaemon.org/technical.html
JIm, WA2ZKD has moved to using spots from wsprdaemon, database wsprnet, table spots - that is data from wsprnet.org obtained via Rob's program and the API. Of course, as today, when wsprnet.org has problems then all users of wsprdaemon spots table, such as WA2ZKD, VK7JJ etc will not have the complete data.
As far as I can see from here with my own spots, there is nothing at all wrong with our own wsprdaemon installations.
I could see this morning that my spots were being accepted into wsprnet.org but not being shown on wsprd.VK7JJ.com for example because the API to bring them in to wsprdaemon, database wsprnet, table spots was timing out - just like our direct queries to wsprnet.org.
Gwyn G3ZIL
Some time ago I put a simplified data flow at the bottom of the webpage at: http://wsprdaemon.org/technical.html
JIm, WA2ZKD has moved to using spots from wsprdaemon, database wsprnet, table spots - that is data from wsprnet.org obtained via Rob's program and the API. Of course, as today, when wsprnet.org has problems then all users of wsprdaemon spots table, such as WA2ZKD, VK7JJ etc will not have the complete data.
As far as I can see from here with my own spots, there is nothing at all wrong with our own wsprdaemon installations.
I could see this morning that my spots were being accepted into wsprnet.org but not being shown on wsprd.VK7JJ.com for example because the API to bring them in to wsprdaemon, database wsprnet, table spots was timing out - just like our direct queries to wsprnet.org.
Gwyn G3ZIL
Rob Robinett
The WD clients independently upload spots to two servers:
1) 'Classic spots' (like those from WSJT-x) are sent directly to wsprnet.org
2) 'Enhanced spots' (the same spots with additional information) to wsprdaemon.org
The WD client caches both sets of spots in the non-volatile directory tree ~/wsprdaemon/uploads.d/..... until the destination server acknowledges successfully receiving those spots. WD limits wsprnet.org upload transactions to at most 1000 spots, so after an extended outage it can take minutes or hours for all of the spots in the WD cache to be uploaded to the wsprnet.org server.
There is a diagram of that data flow at: http://wsprdaemon.org/technical.html
Since Nov 20th Jim's jillil.com 'top spotters' site has obtained its spots from the wsprdaemon.org scrape.
Phil's wsprd.vk7jj.com site and Peter's WsprWatch IOS app also use the wsprdaemon.org scrape.
++++++++++
The WD server scrapes the wsprnet.org database 3 times during each 2 minute wspr cycle. Occasional timeouts are common, but in the last 24 hours the scraper has been frequently reporting multiple consecutive timeouts. The WD scraper reports missing sequence numbers ('gaps') in those spot reports, but in spite of all the timeouts I see relatively few large gaps in the spot sequence numbers once they are finally delivered.
However, neither the WD client nor the WD scraper can verify if the wsprnet.org database is dropping the spots it acknowledges that it has received. It is on my TODO list to add a 'VALIDATE' option to the WD client's wsprnet.org functions. Until then, I know of nowhere to learn if wsprnet.org has accepted but then invisibly dropped the spots being uploaded. I can think of no way for the WD scraper to more fully validate the spot data it receives from its API call to the wsprnet.org server.
Rob
On Sun, Dec 20, 2020 at 9:22 AM ON5KQ <ON5KQ@...> wrote:
Hi Gwyn,
Does WA2ZKD's list reflect the spots in the wsprnet.org site, or does it show wsprdaemon database ?
New spots are showing up now again on the WA2ZKD site as well as on Phil's website wsprd.vk7jj
However where are the older spots from today - only 9000 spots showed up so far - it should be about 20000 instead at this time... Are these spots stored somewhere or lost ?
The reason, why I want to know is, that I do not know, what to monitor is such a case as today. Perhaps I should have taken action, as there is something wrong here with my PC ?
However:
- I know, decoder worked ok
- I know in the afternoon all decodes where stored in many files waiting to be accepted as uploaded. However it doesn't happen. Instead at a certain point all files with old spots from today disappeared and never showed up again... at least not on WA2ZLD site as well as VK7JJ's site.
There is a gap with no spots from 8h28utc to 16h46 utc. with no spots in between, which explains only 9000 spots so far... Does it mean both websites are not showing wsprdeamon database but wsprnet.org, which is faulty ?
Unfortunately I do not understand the data flow.... hope to learn it sometimes...hi
73s
Ulli, ON5KQ
--
ON5KQ
Many thanks for all the explainations... it is clear now.
So as a conclusion from the client point of view:
In any case it is best, to just keep the system running and not to try some "tweek-action" or even reboot at the client site, as soon as it is verified, that wsprd decodes correctly at the client site.
Ulli, ON5KQ
So as a conclusion from the client point of view:
In any case it is best, to just keep the system running and not to try some "tweek-action" or even reboot at the client site, as soon as it is verified, that wsprd decodes correctly at the client site.
Ulli, ON5KQ
Rob Robinett
I have just learned from Gary that wsprnet.org was being subjected to DOS attacks.
On Mon, Dec 21, 2020 at 12:54 AM ON5KQ <ON5KQ@...> wrote:
Many thanks for all the explainations... it is clear now.
So as a conclusion from the client point of view:
In any case it is best, to just keep the system running and not to try some "tweek-action" or even reboot at the client site, as soon as it is verified, that wsprd decodes correctly at the client site.
Ulli, ON5KQ
--