Date   

Re: Intl. WSPR Beacon Project - Grafana Dashboards

Roland
 
Edited

Just a quick heads up, first WSPR Beacon has gone live. IU2PJI is band hopping on 8 Bands. More Beacons will follow very soon
https://www.google.com/maps/d/u/1/edit?mid=14X0GJ4vSQ7D8piZfuHDs902Y9tINuPqB&usp=sharing

73
Roland


Re: How to configure RTL_SDR

Hidehiko Komachi - JA9MAT
 

Thanks Rob quickly reply!

OK all about RTL_SDR issues, I will continue to looking for your progress to resolve it.

"Wsprdaemon" is the best for energy saving wspr receiving.

73 and thanks for nice project, JA9MAT Hidehiko. 


Re: How to configure RTL_SDR

Rob Robinett
 

Welcome Hidehiko,

The RTL-SDR support in WD V2.x is very experimental and even I haven't used it in many months.
'wd -i' is used with baseband AUDIO input devices which is functional but to use it you will need to attach one or more USB audio input dongles.
In WD v3.x I am adding full support for RTL-SDRs and any other SDRs which have ;SoapyAPI' drivers with the primary goal of having WD support the VHF/UHF bands not supported by the Kiwi. 
However I have foudn that the RTL-SDR drivers tune very inaccurately by WSPR requirements.  When I program a GSPDO driven RTL-SDR to tune up 1 Hz, it may jump up 400 Hz or down 200 Hz. 
I suspect that WSPR users of RTL-SDRs are able to fiddle with their tuning and get WSPR decoding to work, but I haven't been able to find a SW algorithm which can do that tuning adjustment.
So until I (or someone else) finds a way to work around that problem, I don't think RTL-SDRs are going to be useful with WD.

Rob


How to configure RTL_SDR

Hidehiko Komachi - JA9MAT
 

Hello,
I began to install and connect two of RTL_SDR at RaspberryPi 3B+.
I configure wsprdaemon.comf to active two of RTL_SDR and run wsprdaemon correctly.
But nothing seems to happen.

So I check audio devices by "./wsprdaemon.sh -i".
Then the following message returned.

---------------------------------
Audio input devices:

**** List of CAPTURE Hardware Devices ****

 

Can't find any audio INPUT devices on this server
--------------------------------

I don't understand the configuration about RTL_SDR deeply.
So if there is a sample wsprdaemon.conf file for RTL_SDR ?
Sorry but have no KiwiSDR yet.

73, JA9MAT Hidehiko.


getting data

Jim Lill
 

I have added this to the data system I maintain. It's for those wanting to do local plotting etc. and not use Grafana or other means

http://jimlill.com:8088/getdata.html/?call=WA2ZKD

change my call to your own 


Re: kfs Omni - signal levels

ON5KQ
 

Hi Rob,
This night, 22 June just after local Midnight in California, I checked the Omni Antenna at kfs again:
- both ports indeed show exactly the same signal (port 8073 and 8083) as it should - as it is the same signal
- the signal level seem to be further reduced as the background noise on 10m today is at -108dBm level, which is ok...  my earlier observation (see above) showed a background noise level on 10m (at the time of dead band) about 6-8db higher...
- no OV clipping indication
- no IMD at the low end frequency (1.8...3Mhz)
- notch arround 3.5Mhz persists...

Last Sunday, I checked the bands on all antennas in the local morning time at earlier daylight hours. I hoped, that noise and interference may be little less than during the week, assuming sourrounding light industrial/commercial activity be more quit on Sunday.
I was shocked however how strong the QRM from broadband noise sources was at about 8h30 in the morning on the high bands especially 12 and 10m... The Noise was at least 10db higher than I have it usually here. That said, I believe my antenna's net gain is probably not less than what the KFS kiwi's see there. I am using a terminated Rhombic on 10m with additionally 17db net preamplifier gain.
Still my noise level is 10db lower !   Noise level on 10m on all antennas at kfs was extremely high (in the -90s dbm !)
I was not prepared to make a detailed report and didn't have the time at Sunday unfortunately, but I will check it again next weekend.

Now at midnight today the noise on 10m is much lower at kfs = there must be something locally radiating like hell to produce that bad reception last Sunday morning (broadband power line noise ?! Influence of local rain shower ?)
May be other local Hams can have a further look to that as well and report the reception on 12m/10m during local daylight hours from kfs...

Greetings to KH6!

Ulli, ON5KQ


Re: 8088 Page restored

Jim Lill
 

For those that have asked about the dupe % etc.   I did some testing and found that simply consumes too much processing time and with spots > 2M would be an issue

On 6/20/21 5:46 AM, Jim Lill wrote:
After a power loss related issue, a few days ago. The header of my Ranking Page is now working again.

-Jim





8088 Page restored

Jim Lill
 

After a power loss related issue, a few days ago. The header of my Ranking Page is now working again.

-Jim


Re: 8 way splitter for GPS to kiwi's and GPSDO needed

Jim Lill
 

ground paths via GPS can be a real issue. I have found that the Time Machine stuff is very bad.

On 6/19/21 2:46 PM, Rob Robinett wrote:
Glen is of course right that it is desirable to not have a GPS splitter introduce ground loops between Kiwis.

However like Steve I have been very successful using simple satellite splitters like this one available from amamzon.fr for 16 euros:


At KPH and at AI6VN/KH6 I put the GPS antenna on the roof and ran 30M of RG-6 to the splitter, then directly connected the splitter output to the Kiwis and have run that way for years without problems.  The RF signal gain from having the antenna on the top of the roof more than makes up for the 12 dB signal loss in the RG-6 and the additional 10+ dB in the splitter.

I use no DC blocking.  Test the RFI level with and without the GPS connections to the Kiwis to see if introducing the splitter adds to your RFI.  If so, then a specialized splitter is of course justified.
   

On Sat, Jun 19, 2021 at 9:59 AM Glenn Elmore <n6gn@...> wrote:
Lots of splitters available but they all end up tieing grounds together which can make a bad situation worse and more complex.
Complete low frequency isolation is desireable but requires special effort.

On Jun 19, 2021 10:31 AM, KD2OM <steve@...> wrote:
Ulli,
I don’t know if an 8 port splitter is available but I am using a 4 port TVRO splitter. The ones meant for satellite TV have one DC thru port and the rest DC blocked. I would imagine the same type is available in Europe. It is 75 ohms with F connectors.

73,
Steve KD2OM 



--
Rob Robinett
AI6VN
mobile: +1 650 218 8896


Re: 8 way splitter for GPS to kiwi's and GPSDO needed

Rob Robinett
 

Glen is of course right that it is desirable to not have a GPS splitter introduce ground loops between Kiwis.

However like Steve I have been very successful using simple satellite splitters like this one available from amamzon.fr for 16 euros:


At KPH and at AI6VN/KH6 I put the GPS antenna on the roof and ran 30M of RG-6 to the splitter, then directly connected the splitter output to the Kiwis and have run that way for years without problems.  The RF signal gain from having the antenna on the top of the roof more than makes up for the 12 dB signal loss in the RG-6 and the additional 10+ dB in the splitter.

I use no DC blocking.  Test the RFI level with and without the GPS connections to the Kiwis to see if introducing the splitter adds to your RFI.  If so, then a specialized splitter is of course justified.
   

On Sat, Jun 19, 2021 at 9:59 AM Glenn Elmore <n6gn@...> wrote:
Lots of splitters available but they all end up tieing grounds together which can make a bad situation worse and more complex.
Complete low frequency isolation is desireable but requires special effort.

On Jun 19, 2021 10:31 AM, KD2OM <steve@...> wrote:
Ulli,
I don’t know if an 8 port splitter is available but I am using a 4 port TVRO splitter. The ones meant for satellite TV have one DC thru port and the rest DC blocked. I would imagine the same type is available in Europe. It is 75 ohms with F connectors.

73,
Steve KD2OM 



--
Rob Robinett
AI6VN
mobile: +1 650 218 8896


Re: 8 way splitter for GPS to kiwi's and GPSDO needed

Glenn Elmore
 

Lots of splitters available but they all end up tieing grounds together which can make a bad situation worse and more complex.
Complete low frequency isolation is desireable but requires special effort.

On Jun 19, 2021 10:31 AM, KD2OM <steve@...> wrote:
Ulli,
I don’t know if an 8 port splitter is available but I am using a 4 port TVRO splitter. The ones meant for satellite TV have one DC thru port and the rest DC blocked. I would imagine the same type is available in Europe. It is 75 ohms with F connectors.

73,
Steve KD2OM 


Re: 8 way splitter for GPS to kiwi's and GPSDO needed

KD2OM
 

Ulli,
I don’t know if an 8 port splitter is available but I am using a 4 port TVRO splitter. The ones meant for satellite TV have one DC thru port and the rest DC blocked. I would imagine the same type is available in Europe. It is 75 ohms with F connectors.

73,
Steve KD2OM 


Re: 8 way splitter for GPS to kiwi's and GPSDO needed

WA2TP
 

Ulli,

I had this same request and brought it to the attention of Makis SV1AFN.

He advised he is working on an 8 port with filtering (an active splitter with 4 or 8 ports, with filters against cell-phones and towers and for all GNSS bands).

I have tied him in so he is aware of the growing need and interest.

Tom
WA2TP 

On Jun 19, 2021, at 11:48 AM, Glenn Elmore <n6gn@...> wrote:



Ulli,

I think you are correct in stating your requirements.  I don't have a complete solution to recommend but I did build a DC isolating GPS board that serves to provide GPS to a single Kiwi without also adding a low frequency path.  For those running multiple Kiwi's from a single GPS source, probably an N-way version of that along with a single DC bias tee or connection to one of the GPS ports is what needs to be constructed. But then there's the question of "what is N?" and 2,4 8 way versions of the thing would need to be fab'd.

It's probably simpler to run dedicated GPS antennas on each Kiwi in situations where a good antenna location is close enough. For the general situation like you describe, an isolating filter is really what is desirable.

Glenn

On 6/19/21 4:59 AM, ON5KQ wrote:
Hi,
Can anyone recommend a 8way splitter to connect my kiwis and a GPSDO for my lab.
What technology I need to look for, to pass DC to the outside antenna but don't (DC-)connect all Kiwis together... I think the kiwis need to be (DC-)isolated to each other

I do not want to import from outside EU however, as current customs costs are horrendous...

Ulli, ON5KQ


Re: 8 way splitter for GPS to kiwi's and GPSDO needed

Glenn Elmore
 

Ulli,

I think you are correct in stating your requirements.  I don't have a complete solution to recommend but I did build a DC isolating GPS board that serves to provide GPS to a single Kiwi without also adding a low frequency path.  For those running multiple Kiwi's from a single GPS source, probably an N-way version of that along with a single DC bias tee or connection to one of the GPS ports is what needs to be constructed. But then there's the question of "what is N?" and 2,4 8 way versions of the thing would need to be fab'd.

It's probably simpler to run dedicated GPS antennas on each Kiwi in situations where a good antenna location is close enough. For the general situation like you describe, an isolating filter is really what is desirable.

Glenn

On 6/19/21 4:59 AM, ON5KQ wrote:
Hi,
Can anyone recommend a 8way splitter to connect my kiwis and a GPSDO for my lab.
What technology I need to look for, to pass DC to the outside antenna but don't (DC-)connect all Kiwis together... I think the kiwis need to be (DC-)isolated to each other

I do not want to import from outside EU however, as current customs costs are horrendous...

Ulli, ON5KQ


8 way splitter for GPS to kiwi's and GPSDO needed

ON5KQ
 

Hi,
Can anyone recommend a 8way splitter to connect my kiwis and a GPSDO for my lab.
What technology I need to look for, to pass DC to the outside antenna but don't (DC-)connect all Kiwis together... I think the kiwis need to be (DC-)isolated to each other

I do not want to import from outside EU however, as current customs costs are horrendous...

Ulli, ON5KQ


Re: One more kfs reception report after attenuators are in position again...

Rob Robinett
 

Thanks Ulli.


On Thu, Jun 17, 2021 at 6:27 AM ON5KQ <ON5KQ@...> wrote:
0h15 (just after midnight California time)

kfs report:
- port 8073
*no OV-indications
*background noise on dead bands (10m,12m...) is too high. Instead of -100dbm noiselevel, in this condition the kiwi-sdr should show -110..-108dbm at maximum (AMwide S-meter reading). As the receiver noise is roughly -115...-117dbm this would make sure that the background noise is at least 6db higher than receiver noise.
Although overloading is not visible is not visible, the bottom 10db sensitivity is useless and available dynamic range should be used for strong signals.
So additional 6db attenuation would improve the situation on low frequencies at night without effecting 10m sensitivity at all during the day.
* slight BC-IMD visible at low frequencies - every 10kHz carriers...
*20db attenuation notch on 3475kHz affects 80m CW, especially weak DX-reception (weak signals cannot be heard!)

-port 8074
*no 0V indication (ADC clipping) visible
*sensitivity on 10m is just right - background noise of -108dbm on 10m is ok
*strong IMD on low frequencies, due to signal distribution fault. Although overall signal strength is lower than on 8073, the IMD is much stronger
*1.8Mhz up to 80m (3.5Mhz) cannot be used due to the IMD at night

-port 8075
*no 0V indication
* background noise level on dead band excellent (-109dbm is just about right)
*no IMD on lowbands
*typical noise from the cellphone cabin is not visible at night although 8843 noise level is low this night (-100dbm at AM-wide)
*unfortunately no 160m...
*overall very good antenna and good set-up

- port 8076
*no 0V indication
* background noise level on dead band excellent (-110dbm is just about right)
*no IMD on lowbands
*unfortunately not usable below 3.5Mhz due to high pass filter
*typical noise from the cellphone cabin is not visible at night, in the 8Mhz band, 8843 noise level is much higher than on 8075 antenna (-89dbm at AM-wide due to high QRN)
*noise level on the low bands is by far the highest on all antennas. I wonder how often the kfs operators switch antennas at all....hi
*very pronounced directivity would be good for hamradio however...

regards,
Ulli, ON5KQ



--
Rob Robinett
AI6VN
mobile: +1 650 218 8896


One more kfs reception report after attenuators are in position again...

ON5KQ
 

0h15 (just after midnight California time)

kfs report:
- port 8073
*no OV-indications
*background noise on dead bands (10m,12m...) is too high. Instead of -100dbm noiselevel, in this condition the kiwi-sdr should show -110..-108dbm at maximum (AMwide S-meter reading). As the receiver noise is roughly -115...-117dbm this would make sure that the background noise is at least 6db higher than receiver noise.
Although overloading is not visible is not visible, the bottom 10db sensitivity is useless and available dynamic range should be used for strong signals.
So additional 6db attenuation would improve the situation on low frequencies at night without effecting 10m sensitivity at all during the day.
* slight BC-IMD visible at low frequencies - every 10kHz carriers...
*20db attenuation notch on 3475kHz affects 80m CW, especially weak DX-reception (weak signals cannot be heard!)

-port 8074
*no 0V indication (ADC clipping) visible
*sensitivity on 10m is just right - background noise of -108dbm on 10m is ok
*strong IMD on low frequencies, due to signal distribution fault. Although overall signal strength is lower than on 8073, the IMD is much stronger
*1.8Mhz up to 80m (3.5Mhz) cannot be used due to the IMD at night

-port 8075
*no 0V indication
* background noise level on dead band excellent (-109dbm is just about right)
*no IMD on lowbands
*typical noise from the cellphone cabin is not visible at night although 8843 noise level is low this night (-100dbm at AM-wide)
*unfortunately no 160m...
*overall very good antenna and good set-up

- port 8076
*no 0V indication
* background noise level on dead band excellent (-110dbm is just about right)
*no IMD on lowbands
*unfortunately not usable below 3.5Mhz due to high pass filter
*typical noise from the cellphone cabin is not visible at night, in the 8Mhz band, 8843 noise level is much higher than on 8075 antenna (-89dbm at AM-wide due to high QRN)
*noise level on the low bands is by far the highest on all antennas. I wonder how often the kfs operators switch antennas at all....hi
*very pronounced directivity would be good for hamradio however...

regards,
Ulli, ON5KQ


Re: What happens with old spots not yet uploaded (probably because of Internet failure) but stored in txt files...

Rob Robinett
 

The wsprnet upload API is not well documented and so I had to experiment with its usage and failure modes.
I found that some batch uploads are accepted by wsprnet, but in its status response wsprnet would say that it accept only some of the spots in the batch (e.g. 190 of 200 spots).
It does not identify which spots were not accepted and if I attempt to reload the whole batch it will reject them all.
Those problems are among the reasons we implemented the wsprdaemon.org extended spots database where we have complete control of the server.
Perhaps someone with more patience and/or influence with wsprnet can resolve this problem, but I think it more productive to work on V3.0

On Mon, Jun 14, 2021 at 5:41 PM WA2TP <myis300@...> wrote:
I too had seen this in the past where wsprdaemon buffered over 3000 spots while my internet was down for several hours, and when the internet connection restored, after ~15 minutes, WSPRNET accepted only 60 of those 3000 buffered spots for some reason. 

Nothing remained in any of the wsprdaemon folders. No idea where the other 2900+ spots went to.

I think I had sent this info to Rob back when this occurred.

Tom
WA2TP 


On Jun 14, 2021, at 8:00 PM, Rob Robinett <rob@...> wrote:


Yes, I think the WD upload service only looks for spot files in receivers which are currently scheduled to be running. That does seem to be a potential source of late uploaded spots, although spots are never flushed until they have been successfully uploaded.   I have added working on that to the v3.0 TODO list.


I don’t know how wsprnet handles such spots, although Glenn has reported success with uploads delayed by hours or a day.  wsprnet assigns a unique 64 bit number to each spot it receives, and in cloning the wsprnet DB I validate that there are no gaps in the spot number sequence.

On Mon, Jun 14, 2021 at 8:17 AM ON5KQ <ON5KQ@...> wrote:
Dear,
During last night I probably had an Internet connection failuer, so from 20h utc yesterday until 4h utc in the morning, there was nothing uploaded.
The wsprdaemon script created a lot of txt files instead (containing the decoded spots during that time). In the meantime the wsprdaemon.conf file switched from nighttime to daylight schedule and a little later during the new daylight schedule the Internet came back.

Because of the new daylight schedule running, the wsprdaemon script didn't realize , that from the nighttime schedule (not active anymore) there were still a lot of ,txt files containing the spots to be uploaded. 
If I remember correctly all the .txt files with the nighttime spots were in this directory
/home/wsprdaemon/uploads.d/wsprnet.d/spots.d/ ....   and channel by channel the txt files were written for every 2min of transmission.

I was wondering, why the waiting spots were not uploaded immediately as the Internet came back in the morning...

To find out,  I restarted with "./wsprdaemon.sh -vvva"  to force high verbosity to check the various log files.
I found that the script searched for txt files to be uploaded, but not in the nighttime folder, but the current job folder for the running daytime schedule - so it would not immediately find the hundreds of waiting txt files only after roughly another 12h, when the night schedule becomes active again.

However, I didn't want to wait that long so...
(I am not sure, if it was correct, what I did then....):
- to upload the waiting txt files immediately, I stopped wsprdaemon with
./wsprdaemon.sh -z
then changed wsprdaemon.conf in that way, that the nighttime schedule was resumed instead of starting the daytime schedule.
Started again with:
./wsprdaemon.sh -vvva

Now In the logs I could see, that immediately more than 300 txt files waiting for upload were found from the script
Quickly the logs also showed that after a while many many uploads were done - it must be thousands of spots...

After a while, all txt files were erased and I found in the curl-logs the confirmation of upload acceptance... again thousands. I was wondering however that all that was only a matter of minutes - the missing upload was very quick...

So, I was happy and stopped wsprdaemon, as I did not want to write too much data in the log-files with that high verbosity level.
So, I restarted with
./wsprdaemon.sh -a

Now what::
Hours later now, still there are no spots at all between 22h yesterday and 6h this morning - neither on wsprnet.org, nor on vk7jj's website...

It would be great to understand what really happened with the uploaded spots from partly today and partly from yesterday (based on utc time)

Will the missing spots ever appear ?
Why I got a confirmation of acceptance for thousands of spots in the curl-file, when these spots never appear ?
What did I wrong, that still no missing spot is delivered and how should I proceed next time to prevent data loss.

Or do just need to wait longer for the data to appear (may be one day or so ?!)
I am confused...hi

May be Rob can explain in the next zoom-meeting, how the upload of old spots really works - either because of Internet connection problems or even after portable kiwi operation in the field with later upload from home (Glenn's application ?!)

regards,
Ulli, ON5KQ



--
Rob Robinett
AI6VN
mobile: +1 650 218 8896


Re: What happens with old spots not yet uploaded (probably because of Internet failure) but stored in txt files...

WA2TP
 

I too had seen this in the past where wsprdaemon buffered over 3000 spots while my internet was down for several hours, and when the internet connection restored, after ~15 minutes, WSPRNET accepted only 60 of those 3000 buffered spots for some reason. 

Nothing remained in any of the wsprdaemon folders. No idea where the other 2900+ spots went to.

I think I had sent this info to Rob back when this occurred.

Tom
WA2TP 


On Jun 14, 2021, at 8:00 PM, Rob Robinett <rob@...> wrote:


Yes, I think the WD upload service only looks for spot files in receivers which are currently scheduled to be running. That does seem to be a potential source of late uploaded spots, although spots are never flushed until they have been successfully uploaded.   I have added working on that to the v3.0 TODO list.


I don’t know how wsprnet handles such spots, although Glenn has reported success with uploads delayed by hours or a day.  wsprnet assigns a unique 64 bit number to each spot it receives, and in cloning the wsprnet DB I validate that there are no gaps in the spot number sequence.

On Mon, Jun 14, 2021 at 8:17 AM ON5KQ <ON5KQ@...> wrote:
Dear,
During last night I probably had an Internet connection failuer, so from 20h utc yesterday until 4h utc in the morning, there was nothing uploaded.
The wsprdaemon script created a lot of txt files instead (containing the decoded spots during that time). In the meantime the wsprdaemon.conf file switched from nighttime to daylight schedule and a little later during the new daylight schedule the Internet came back.

Because of the new daylight schedule running, the wsprdaemon script didn't realize , that from the nighttime schedule (not active anymore) there were still a lot of ,txt files containing the spots to be uploaded. 
If I remember correctly all the .txt files with the nighttime spots were in this directory
/home/wsprdaemon/uploads.d/wsprnet.d/spots.d/ ....   and channel by channel the txt files were written for every 2min of transmission.

I was wondering, why the waiting spots were not uploaded immediately as the Internet came back in the morning...

To find out,  I restarted with "./wsprdaemon.sh -vvva"  to force high verbosity to check the various log files.
I found that the script searched for txt files to be uploaded, but not in the nighttime folder, but the current job folder for the running daytime schedule - so it would not immediately find the hundreds of waiting txt files only after roughly another 12h, when the night schedule becomes active again.

However, I didn't want to wait that long so...
(I am not sure, if it was correct, what I did then....):
- to upload the waiting txt files immediately, I stopped wsprdaemon with
./wsprdaemon.sh -z
then changed wsprdaemon.conf in that way, that the nighttime schedule was resumed instead of starting the daytime schedule.
Started again with:
./wsprdaemon.sh -vvva

Now In the logs I could see, that immediately more than 300 txt files waiting for upload were found from the script
Quickly the logs also showed that after a while many many uploads were done - it must be thousands of spots...

After a while, all txt files were erased and I found in the curl-logs the confirmation of upload acceptance... again thousands. I was wondering however that all that was only a matter of minutes - the missing upload was very quick...

So, I was happy and stopped wsprdaemon, as I did not want to write too much data in the log-files with that high verbosity level.
So, I restarted with
./wsprdaemon.sh -a

Now what::
Hours later now, still there are no spots at all between 22h yesterday and 6h this morning - neither on wsprnet.org, nor on vk7jj's website...

It would be great to understand what really happened with the uploaded spots from partly today and partly from yesterday (based on utc time)

Will the missing spots ever appear ?
Why I got a confirmation of acceptance for thousands of spots in the curl-file, when these spots never appear ?
What did I wrong, that still no missing spot is delivered and how should I proceed next time to prevent data loss.

Or do just need to wait longer for the data to appear (may be one day or so ?!)
I am confused...hi

May be Rob can explain in the next zoom-meeting, how the upload of old spots really works - either because of Internet connection problems or even after portable kiwi operation in the field with later upload from home (Glenn's application ?!)

regards,
Ulli, ON5KQ


Re: What happens with old spots not yet uploaded (probably because of Internet failure) but stored in txt files...

Rob Robinett
 

Yes, I think the WD upload service only looks for spot files in receivers which are currently scheduled to be running. That does seem to be a potential source of late uploaded spots, although spots are never flushed until they have been successfully uploaded.   I have added working on that to the v3.0 TODO list.


I don’t know how wsprnet handles such spots, although Glenn has reported success with uploads delayed by hours or a day.  wsprnet assigns a unique 64 bit number to each spot it receives, and in cloning the wsprnet DB I validate that there are no gaps in the spot number sequence.

On Mon, Jun 14, 2021 at 8:17 AM ON5KQ <ON5KQ@...> wrote:
Dear,
During last night I probably had an Internet connection failuer, so from 20h utc yesterday until 4h utc in the morning, there was nothing uploaded.
The wsprdaemon script created a lot of txt files instead (containing the decoded spots during that time). In the meantime the wsprdaemon.conf file switched from nighttime to daylight schedule and a little later during the new daylight schedule the Internet came back.

Because of the new daylight schedule running, the wsprdaemon script didn't realize , that from the nighttime schedule (not active anymore) there were still a lot of ,txt files containing the spots to be uploaded. 
If I remember correctly all the .txt files with the nighttime spots were in this directory
/home/wsprdaemon/uploads.d/wsprnet.d/spots.d/ ....   and channel by channel the txt files were written for every 2min of transmission.

I was wondering, why the waiting spots were not uploaded immediately as the Internet came back in the morning...

To find out,  I restarted with "./wsprdaemon.sh -vvva"  to force high verbosity to check the various log files.
I found that the script searched for txt files to be uploaded, but not in the nighttime folder, but the current job folder for the running daytime schedule - so it would not immediately find the hundreds of waiting txt files only after roughly another 12h, when the night schedule becomes active again.

However, I didn't want to wait that long so...
(I am not sure, if it was correct, what I did then....):
- to upload the waiting txt files immediately, I stopped wsprdaemon with
./wsprdaemon.sh -z
then changed wsprdaemon.conf in that way, that the nighttime schedule was resumed instead of starting the daytime schedule.
Started again with:
./wsprdaemon.sh -vvva

Now In the logs I could see, that immediately more than 300 txt files waiting for upload were found from the script
Quickly the logs also showed that after a while many many uploads were done - it must be thousands of spots...

After a while, all txt files were erased and I found in the curl-logs the confirmation of upload acceptance... again thousands. I was wondering however that all that was only a matter of minutes - the missing upload was very quick...

So, I was happy and stopped wsprdaemon, as I did not want to write too much data in the log-files with that high verbosity level.
So, I restarted with
./wsprdaemon.sh -a

Now what::
Hours later now, still there are no spots at all between 22h yesterday and 6h this morning - neither on wsprnet.org, nor on vk7jj's website...

It would be great to understand what really happened with the uploaded spots from partly today and partly from yesterday (based on utc time)

Will the missing spots ever appear ?
Why I got a confirmation of acceptance for thousands of spots in the curl-file, when these spots never appear ?
What did I wrong, that still no missing spot is delivered and how should I proceed next time to prevent data loss.

Or do just need to wait longer for the data to appear (may be one day or so ?!)
I am confused...hi

May be Rob can explain in the next zoom-meeting, how the upload of old spots really works - either because of Internet connection problems or even after portable kiwi operation in the field with later upload from home (Glenn's application ?!)

regards,
Ulli, ON5KQ

201 - 220 of 508