|
Re: What happens with old spots not yet uploaded (probably because of Internet failure) but stored in txt files...
WA2TP - Tom
I too had seen this in the past where wsprdaemon buffered over 3000 spots while my internet was down for several hours, and when the internet connection restored, after ~15 minutes, WSPRNET accepted only 60 of those 3000 buffered spots for some reason.
toggle quoted messageShow quoted text
Nothing remained in any of the wsprdaemon folders. No idea where the other 2900+ spots went to.
I think I had sent this info to Rob back when this occurred.
Tom
WA2TP
On Jun 14, 2021, at 8:00 PM, Rob Robinett <rob@...> wrote:
|
|
|
|
Re: What happens with old spots not yet uploaded (probably because of Internet failure) but stored in txt files...
Rob Robinett
Yes, I think the WD upload service only looks for spot files in receivers which are currently scheduled to be running. That does seem to be a potential source of late uploaded spots, although spots are never flushed until they have been successfully uploaded. I have added working on that to the v3.0 TODO list. I don’t know how wsprnet handles such spots, although Glenn has reported success with uploads delayed by hours or a day. wsprnet assigns a unique 64 bit number to each spot it receives, and in cloning the wsprnet DB I validate that there are no gaps in the spot number sequence.
On Mon, Jun 14, 2021 at 8:17 AM ON5KQ <ON5KQ@...> wrote: Dear,
|
|
|
|
What happens with old spots not yet uploaded (probably because of Internet failure) but stored in txt files...
ON5KQ
Dear,
During last night I probably had an Internet connection failuer, so from 20h utc yesterday until 4h utc in the morning, there was nothing uploaded. The wsprdaemon script created a lot of txt files instead (containing the decoded spots during that time). In the meantime the wsprdaemon.conf file switched from nighttime to daylight schedule and a little later during the new daylight schedule the Internet came back. Because of the new daylight schedule running, the wsprdaemon script didn't realize , that from the nighttime schedule (not active anymore) there were still a lot of ,txt files containing the spots to be uploaded. If I remember correctly all the .txt files with the nighttime spots were in this directory /home/wsprdaemon/uploads.d/wsprnet.d/spots.d/ .... and channel by channel the txt files were written for every 2min of transmission. I was wondering, why the waiting spots were not uploaded immediately as the Internet came back in the morning... To find out, I restarted with "./wsprdaemon.sh -vvva" to force high verbosity to check the various log files. I found that the script searched for txt files to be uploaded, but not in the nighttime folder, but the current job folder for the running daytime schedule - so it would not immediately find the hundreds of waiting txt files only after roughly another 12h, when the night schedule becomes active again. However, I didn't want to wait that long so... (I am not sure, if it was correct, what I did then....): - to upload the waiting txt files immediately, I stopped wsprdaemon with ./wsprdaemon.sh -z then changed wsprdaemon.conf in that way, that the nighttime schedule was resumed instead of starting the daytime schedule. Started again with: ./wsprdaemon.sh -vvva Now In the logs I could see, that immediately more than 300 txt files waiting for upload were found from the script Quickly the logs also showed that after a while many many uploads were done - it must be thousands of spots... After a while, all txt files were erased and I found in the curl-logs the confirmation of upload acceptance... again thousands. I was wondering however that all that was only a matter of minutes - the missing upload was very quick... So, I was happy and stopped wsprdaemon, as I did not want to write too much data in the log-files with that high verbosity level. So, I restarted with ./wsprdaemon.sh -a Now what:: Hours later now, still there are no spots at all between 22h yesterday and 6h this morning - neither on wsprnet.org, nor on vk7jj's website... It would be great to understand what really happened with the uploaded spots from partly today and partly from yesterday (based on utc time) Will the missing spots ever appear ? Why I got a confirmation of acceptance for thousands of spots in the curl-file, when these spots never appear ? What did I wrong, that still no missing spot is delivered and how should I proceed next time to prevent data loss. Or do just need to wait longer for the data to appear (may be one day or so ?!) I am confused...hi May be Rob can explain in the next zoom-meeting, how the upload of old spots really works - either because of Internet connection problems or even after portable kiwi operation in the field with later upload from home (Glenn's application ?!) regards, Ulli, ON5KQ
|
|
|
|
Re: first time more than 3000spots/hour on all bands...
ON5KQ
I mean the current one, which wsprdaemon is using. However I remember the last update of wsprd decoder engine, it was said by the developers, that it should be more efficient in busy bands. Also the summer QRN is a real stress test for the decoder. The developers said, that also in summer qrn their latest decoder (which we have now for at least 6 month or so) should work better....
So nothing to do...hi Yesterday was just a good test case to verify the robustness of the decoder - I think we must be happy to have such good software. Today I received my first UK-station on 4m (70Mhz) over a distance of a little more than 450km - no special propagation just groundwave... The last couple of days I am testing how to use my terminated rhombic antenna on 6m and above. It is pointing to the UK and it turns out to be very broadband. I use it from 160m to 70Mhz for reception with good results. VSWR as measured with my DG8SAQ VNA: (30kHz to 60Mhz trace)  Not that bad... Ulli, ON5KQ
|
|
|
|
Re: first time more than 3000spots/hour on all bands...
WA2TP - Tom
Ulli,
toggle quoted messageShow quoted text
Which new decoder are you referring to?
On Jun 11, 2021, at 2:40 AM, ON5KQ <ON5KQ@...> wrote:
|
|
|
|
Re: first time more than 3000spots/hour on all bands...
ON5KQ
Jim,
on 20m (and the higher bands) usually I hear the locals (100km distance) best, when both antennas are in the same direction (back-scatter) So in the evening in 20m CW or SSB if stations put their beams to Northamerica, I would hear them best with the beam to Northamerica too. Although the many Germans heard like this are all in the back of the antenna... So, I think the usually not very directive Wspr-wire antennas (both TX and RX, compared to a Yagi beam antenna) will work in the same way: Backscatter signals are usually stronger than direct ground wave propagation. Ulli
|
|
|
|
Re: first time more than 3000spots/hour on all bands...
ON5KQ
yes, you can filter.... Phil's website (VK7JJ) offers the possibility.
Obviously, due to the short skip "QRM" the reception capability for the much weaker DX stations is limited. When there is so much short skip sporadic E reception often F2 reflections (being the normal propagation on shortwave) is not possible or attenuated due to absorbtion. Yesterday the 20m band opened to North-America after the short skip signals dropped in signal strength. I was surprised, how well the latest decoder really can cope with extremely busy bands when such propagation overloads the available wspr-frequency space with transmitting stations. I must say it is impressive, that the decoder can reliably pick more than 40stations in a 2min wspr circle from just 200Hz wide band... How far this will develop in the coming years, when we probably will see a fully open 10m band even during the night and at the same time have good condx on the low bands ? That is the moment, where I need a new PC for decoding...hi. The current solution with a dual CPU XEON workstation might be not enough.. The sporadic E max usable frequency yesterday was 47Mhz at about 21h (at the time of my first post above) according to the Ionosonde here in Belgium. Still not enough for opening on 6m although the regular stations were heard via tropo, I guess. Ulli
|
|
|
|
Re: KFS reception quality - IMD and other (internal?) noise sources...
Rob Robinett
I have just returned from 5 hours at KFS and have time for only a brief report: 1) I removed the Nooelec AMB BF and 6/9 dB pads which were between the Qbit LNA outputs and the Kiwi RF inputs. There are 2 Mhz HPF out at the antennas and I have muted the whole of the AM band, so no user can tune there on a Kiwi. So the Omni Kiwis have additional 6 dB gain and the others an additional 9 dB gain. I will monitor WD's overload log to verify those changes don't result in significant overload events. 2) I confirmed that the NW feed from the primary multicoupler is suffering from severe AM IMD in the 2-4 Mhz region, so this IMB is generated in that multicoupler or more likely by the LNA out at the antenna Omni A and B also show that IMD but to much lesser extent. Fixing these will require help from ARINC 3) Thanks to Glenn's AA and the tinySA, I tracked down the source of the every 40 Khz switcher RFI: it is a vertically polarized signal generated at the cell phone equipment shed at the base of the cell phone tower shared with the NW antenna. Again, ARINC is going to have to get that fixed if they care about it. That same RFI can be heard on the Omni which is the other antenna closest to the cell phone shed (about 100M away). The SE and SW antennas are about 200+M away from that shed and so that RFI is barely noticable on those antennas. 4) I see other switcher RFIs, so I'm sure there are many other bugs to squash, but the signals we see on the Kiwis are pretty much what the portable Kiwi sees when connected to the primary multicouplers. So all indications are the problems are outside the building. 4) Glen''s AA + the portable Kiwi showed a very clean spectrum when I got 300M away from the cell shed and main building. There is very good wifi coverage out in the fields, so I think deploying a solar powered AA + Kiwi as far as possible (e.g 400M) from the cell shed would result in a very clean full spectrum. I now understand the location of all the antennas and can see why the NW and Omni will always be impaired by the cell shed. Rob
On Thu, Jun 10, 2021 at 12:33 PM ON5KQ <ON5KQ@...> wrote: In the meantime, I must correct my report from above. There are indeed also during the night severe man-made noise sources, which I believe are produced somewhere in the signal distribution system.,,,,more detailed reports I have sent to Rob, to help to find a solution and support kfs... --
|
|
|
|
Re: WSPRDaemon creating empty spot files and nothing for WSPRNet
Rob Robinett
Duncan gave me ssh access to his Pi and I was able to see that his RaspSDR wasn't sending audio through the kiwirecorder session, nor was autowspr able to decode on the RaspSDR. Duncan later reported that an update to his RaspSDR SW fixed those problems and WD was running fine. However I don't know any more detail about the RaspSDR SW change.
On Thu, Jun 10, 2021 at 1:33 PM Jim Lill <jim@...> wrote:
--
|
|
|
|
Re: first time more than 3000spots/hour on all bands...
WA2TP - Tom
Perhaps there is way to filter out by distance and disregard spots that are under xxx Km?
From: wsprdaemon@groups.io <wsprdaemon@groups.io> on behalf of Jim Lill <jim@...>
Sent: Thursday, June 10, 2021 4:30 PM To: wsprdaemon@groups.io <wsprdaemon@groups.io> Subject: Re: [wsprdaemon] first time more than 3000spots/hour on all bands... Looking at the data, I see that the EU advantage over North America is on 17-10m While I'm sure that much of that is from E and F skip I'd bet there is some GW and Ground Scatter in there too. When bands are "supposed to be hot" more people turn there
TX on and those shorter paths in EU have those TX to take advantage of. When I have time, I'll try to do some analysis on that -Jim WA2ZKD On 6/10/21 3:48 PM, KD2OM wrote:
We US stations might as well pack up our tents and go home:) As soon as the cycle moves along we will be lucky to make the top ten World Wide.
|
|
|
|
Re: WSPRDaemon creating empty spot files and nothing for WSPRNet
Jim Lill
does wsprd decoder ever run?
On 6/9/21 7:45 PM, Duncan GM5JET wrote:
Hi Rob,
|
|
|
|
Re: first time more than 3000spots/hour on all bands...
Jim Lill
Looking at the data, I see that the EU advantage over North
America is on 17-10m While I'm sure that much of that is from E
and F skip I'd bet there is some GW and Ground Scatter in there
too. When bands are "supposed to be hot" more people turn there
TX on and those shorter paths in EU have those TX to take
advantage of. When I have time, I'll try to do some analysis on that -Jim WA2ZKD
On 6/10/21 3:48 PM, KD2OM wrote:
We US stations might as well pack up our tents and go home:) As soon as the cycle moves along we will be lucky to make the top ten World Wide.
|
|
|
|
Re: first time more than 3000spots/hour on all bands...
We US stations might as well pack up our tents and go home:) As soon as the cycle moves along we will be lucky to make the top ten World Wide.
toggle quoted messageShow quoted text
73 Steve KD2OM .
On Jun 10, 2021, at 15:40, ON5KQ <ON5KQ@...> wrote:
|
|
|
|
Re: first time more than 3000spots/hour on all bands...
ON5KQ
just checked: 20m only spots frequently about 40stations per 2min... currently at 900spots/hour - incredible and only possible with the current decoder, I guess - I don't think earlier versions were able to handle such overcrowded wspr-channels...
Ulli, ON5KQ
|
|
|
|
Re: KFS reception quality - IMD and other (internal?) noise sources...
ON5KQ
In the meantime, I must correct my report from above. There are indeed also during the night severe man-made noise sources, which I believe are produced somewhere in the signal distribution system.,,,,more detailed reports I have sent to Rob, to help to find a solution and support kfs...
|
|
|
|
first time more than 3000spots/hour on all bands...
ON5KQ
Short skip conditions here in Europe make my 2x Xeon quadcore Workstation be real busy for the first time...
300km far away stations on 20m are 40db over S9 on the rhombic to northwest. This antenna favours rather high angle radiation on 20m.... so the short skip propagation put the kiwi's on fire...hi 10db att-pads nessesary for the first time as well this evening... I saw Holgers kiwi also overload for short moments for the strong UK BC-signals from Wooferton (250...500kW on 20dbi ant) My kiwi shows S9+55db for these stations today... about 19h utc, 21h local, local sunset expected at 19h58 in roughly one hour.... the computer needs almost 70sec - never experienced before Frank LX1DQ also spots really al lot (also almost 3000spots all band/hour), Holger similar All bands full throttle (incl. 10m) at the moment... Only 6m suffers now, due to another 6db att.pad needed to prevent overloading at the rhombic to northwest ... I think within 30min I can remove it again, as signal strength will weaker a bit... 
|
|
|
|
Has anyone run auto-wspr and/or WD on a RaspSDR?
Rob Robinett
One of our users is tying to use a RaspSDR and neither auto-wspr nor WD are seeing signals which are present on the main waterfall display.
|
|
|
|
Re: WSPRDaemon creating empty spot files and nothing for WSPRNet
Duncan GM5JET
Details sent via PM
|
|
|
|
Re: WSPRDaemon creating empty spot files and nothing for WSPRNet
Rob Robinett
Can you give me ssh access to your Pi?
Or we could do a Zoom session
|
|
|
|
Re: WSPRDaemon creating empty spot files and nothing for WSPRNet
Duncan GM5JET
Hi Rob,
There is only the entry in the log file for starting the pid and then nothing else. I have watched the updates and I get the message that states that the files contained no spots and that the files should be flushed. I'm rather stumped! 73 Duncan
|
|
|