Date   

Re: Second KIWI setup?

Rob Robinett
 

WD fully supports getting spots for 2 or more Kiwis, selecting one set of the best received spots and uploading only one copy of each spot under the single receiver call sign to wsprnet.org.
 
In addition, WD creates a log file where all spots are listed and a comparison of the SNR of the spots is listed.
Further, all spots from all receives can be recorded in the wsprdaemon.org SQL database for further analysis and text and graphs using SQL and Grafana.
Download WD and you will find in the config file a template for MERGE receivers.

Rob


Re: Second KIWI setup?

Edward Hammond
 

On 1/5/21 5:59 PM, John via groups.io wrote:

I am not clear how to use the "Merge" assignments. I want to avoid double listing the same spots.
I'm interested in similar antenna experiments, and have the same question about avoiding duplicate submissions.

Of course I could put a suffix on the callsign of one of the Kiwis, but it would be preferable to report both under the same call.

I fear, though, that there's no easy answer.

EH


Re: Second KIWI setup?

Rob Robinett
 

It appears that all of your rx channels are the same  all day, so your config file can be much simpler.
mail your config file and I'll fix it up.

On Tue, Jan 5, 2021 at 3:59 PM John via groups.io <n0ure=yahoo.com@groups.io> wrote:
My first KIWI is using 8 bands of vertical 10M -80M. Mu wsprdaemon.conf lists:
        "KIWI_0                  192.168.2.160:8073     TI4JWC         EK70wb   
and
 
declare WSPR_SCHEDULE_complex=(
        "06:00  KIWI_0,80  KIWI_0,40 KIWI_0,30 KIWI_0,20 KIWI_0,17 KIWI_0,15 KIWI_0,12 KIWI_0,10"
        "07:00  KIWI_0,80  KIWI_0,40 KIWI_0,30 KIWI_0,20 KIWI_0,17 KIWI_0,15 KIWI_0,12 KIWI_0,10"
        "08:00  KIWI_0,80  KIWI_0,40 KIWI_0,30 KIWI_0,20 KIWI_0,17 KIWI_0,15 KIWI_0,12 KIWI_0,10"
        "09:00  KIWI_0,80  KIWI_0,40 KIWI_0,30 KIWI_0,20 KIWI_0,17 KIWI_0,15 KIWI_0,12 KIWI_0,10"
        "10:00  KIWI_0,80  KIWI_0,40 KIWI_0,30 KIWI_0,20 KIWI_0,17 KIWI_0,15 KIWI_0,12 KIWI_0,10"
        "11:00  KIWI_0,80  KIWI_0,40 KIWI_0,30 KIWI_0,20 KIWI_0,17 KIWI_0,15 KIWI_0,12 KIWI_0,10"
        "12:00  KIWI_0,80  KIWI_0,40 KIWI_0,30 KIWI_0,20 KIWI_0,17 KIWI_0,15 KIWI_0,12 KIWI_0,10"
        "13:00  KIWI_0,80  KIWI_0,40 KIWI_0,30 KIWI_0,20 KIWI_0,17 KIWI_0,15 KIWI_0,12 KIWI_0,10"
        "14:00  KIWI_0,80  KIWI_0,40 KIWI_0,30 KIWI_0,20 KIWI_0,17 KIWI_0,15 KIWI_0,12 KIWI_0,10"
        "15:00  KIWI_0,80  KIWI_0,40 KIWI_0,30 KIWI_0,20 KIWI_0,17 KIWI_0,15 KIWI_0,12 KIWI_0,10"...................................

For the schedule.

For the second KIWI on the Hex beam I think this will be the new arangement.
declare RECEIVER_LIST=(
        "KIWI_0                  192.168.2.160:8073     TI4JWC         EK70wb    
         KIWI_1                  192.168.2.161:8073     TI4JWC         EK70WB  
and
 
declare WSPR_SCHEDULE_complex=(
        "06:00  KIWI_0,80  KIWI_0,40 KIWI_0,30 KIWI_0,20 KIWI_0,17 KIWI_0,15 KIWI_0,12 KIWI_0,10
                KIWI_1,20  KIWI_1,17 KIWI_1,15 KIWI_1,12 KIWI_1,10 "
        "07:00  KIWI_0,80  KIWI_0,40 KIWI_0,30 KIWI_0,20 KIWI_0,17 KIWI_0,15 KIWI_0,12 KIWI_0,10
                KIWI_1,20  KIWI_1,17 KIWI_1,15 KIWI_1,12 KIWI_1,10 "
        "08:00  KIWI_0,80  KIWI_0,40 KIWI_0,30 KIWI_0,20 KIWI_0,17 KIWI_0,15 KIWI_0,12 KIWI_0,10
                KIWI_1,20  KIWI_1,17 KIWI_1,15 KIWI_1,12 KIWI_1,10 "
        "09:00  KIWI_0,80  KIWI_0,40 KIWI_0,30 KIWI_0,20 KIWI_0,17 KIWI_0,15 KIWI_0,12 KIWI_0,10
                KIWI_1,20  KIWI_1,17 KIWI_1,15 KIWI_1,12 KIWI_1,10 "
        "10:00  KIWI_0,80  KIWI_0,40 KIWI_0,30 KIWI_0,20 KIWI_0,17 KIWI_0,15 KIWI_0,12 KIWI_0,10
                KIWI_1,20  KIWI_1,17 KIWI_1,15 KIWI_1,12 KIWI_1,10 "
        "11:00  KIWI_0,80  KIWI_0,40 KIWI_0,30 KIWI_0,20 KIWI_0,17 KIWI_0,15 KIWI_0,12 KIWI_0,10
                KIWI_1,20  KIWI_1,17 KIWI_1,15 KIWI_1,12 KIWI_1,10 "
        "12:00  KIWI_0,80  KIWI_0,40 KIWI_0,30 KIWI_0,20 KIWI_0,17 KIWI_0,15 KIWI_0,12 KIWI_0,10

for the combined schedule.
I am not clear how to use the "Merge" assignments. I want to avoid double listing the same spots.

John
TI4JWC
  



--
Rob Robinett
AI6VN
mobile: +1 650 218 8896


Second KIWI setup?

John
 

My first KIWI is using 8 bands of vertical 10M -80M. Mu wsprdaemon.conf lists:
        "KIWI_0                  192.168.2.160:8073     TI4JWC         EK70wb   
and
 
declare WSPR_SCHEDULE_complex=(
        "06:00  KIWI_0,80  KIWI_0,40 KIWI_0,30 KIWI_0,20 KIWI_0,17 KIWI_0,15 KIWI_0,12 KIWI_0,10"
        "07:00  KIWI_0,80  KIWI_0,40 KIWI_0,30 KIWI_0,20 KIWI_0,17 KIWI_0,15 KIWI_0,12 KIWI_0,10"
        "08:00  KIWI_0,80  KIWI_0,40 KIWI_0,30 KIWI_0,20 KIWI_0,17 KIWI_0,15 KIWI_0,12 KIWI_0,10"
        "09:00  KIWI_0,80  KIWI_0,40 KIWI_0,30 KIWI_0,20 KIWI_0,17 KIWI_0,15 KIWI_0,12 KIWI_0,10"
        "10:00  KIWI_0,80  KIWI_0,40 KIWI_0,30 KIWI_0,20 KIWI_0,17 KIWI_0,15 KIWI_0,12 KIWI_0,10"
        "11:00  KIWI_0,80  KIWI_0,40 KIWI_0,30 KIWI_0,20 KIWI_0,17 KIWI_0,15 KIWI_0,12 KIWI_0,10"
        "12:00  KIWI_0,80  KIWI_0,40 KIWI_0,30 KIWI_0,20 KIWI_0,17 KIWI_0,15 KIWI_0,12 KIWI_0,10"
        "13:00  KIWI_0,80  KIWI_0,40 KIWI_0,30 KIWI_0,20 KIWI_0,17 KIWI_0,15 KIWI_0,12 KIWI_0,10"
        "14:00  KIWI_0,80  KIWI_0,40 KIWI_0,30 KIWI_0,20 KIWI_0,17 KIWI_0,15 KIWI_0,12 KIWI_0,10"
        "15:00  KIWI_0,80  KIWI_0,40 KIWI_0,30 KIWI_0,20 KIWI_0,17 KIWI_0,15 KIWI_0,12 KIWI_0,10"...................................

For the schedule.

For the second KIWI on the Hex beam I think this will be the new arangement.
declare RECEIVER_LIST=(
        "KIWI_0                  192.168.2.160:8073     TI4JWC         EK70wb    
         KIWI_1                  192.168.2.161:8073     TI4JWC         EK70WB  
and
 
declare WSPR_SCHEDULE_complex=(
        "06:00  KIWI_0,80  KIWI_0,40 KIWI_0,30 KIWI_0,20 KIWI_0,17 KIWI_0,15 KIWI_0,12 KIWI_0,10
                KIWI_1,20  KIWI_1,17 KIWI_1,15 KIWI_1,12 KIWI_1,10 "
        "07:00  KIWI_0,80  KIWI_0,40 KIWI_0,30 KIWI_0,20 KIWI_0,17 KIWI_0,15 KIWI_0,12 KIWI_0,10
                KIWI_1,20  KIWI_1,17 KIWI_1,15 KIWI_1,12 KIWI_1,10 "
        "08:00  KIWI_0,80  KIWI_0,40 KIWI_0,30 KIWI_0,20 KIWI_0,17 KIWI_0,15 KIWI_0,12 KIWI_0,10
                KIWI_1,20  KIWI_1,17 KIWI_1,15 KIWI_1,12 KIWI_1,10 "
        "09:00  KIWI_0,80  KIWI_0,40 KIWI_0,30 KIWI_0,20 KIWI_0,17 KIWI_0,15 KIWI_0,12 KIWI_0,10
                KIWI_1,20  KIWI_1,17 KIWI_1,15 KIWI_1,12 KIWI_1,10 "
        "10:00  KIWI_0,80  KIWI_0,40 KIWI_0,30 KIWI_0,20 KIWI_0,17 KIWI_0,15 KIWI_0,12 KIWI_0,10
                KIWI_1,20  KIWI_1,17 KIWI_1,15 KIWI_1,12 KIWI_1,10 "
        "11:00  KIWI_0,80  KIWI_0,40 KIWI_0,30 KIWI_0,20 KIWI_0,17 KIWI_0,15 KIWI_0,12 KIWI_0,10
                KIWI_1,20  KIWI_1,17 KIWI_1,15 KIWI_1,12 KIWI_1,10 "
        "12:00  KIWI_0,80  KIWI_0,40 KIWI_0,30 KIWI_0,20 KIWI_0,17 KIWI_0,15 KIWI_0,12 KIWI_0,10

for the combined schedule.
I am not clear how to use the "Merge" assignments. I want to avoid double listing the same spots.

John
TI4JWC
  


New to WSPR Daemon

Greg Beam <ki7mt01@...>
 

Hello All,

I just recently found WSPR Daemon ( Very Cool !! ), and I'm very interested in how this will progress over the next SS cycle. I had  started a similar project (on a much, much smaller scale) for WSPRnet data. I am (was) mainly focused on using Spark (Scala), PySpark, Apache Arrow, and various utilities to convert those monster CSV files to Parquet / Avro  formats; not so much on the visualization ( but that may change now :-) )

I have truck-load of content to add, but, I did start working on a db-init-script for a local PostgreSQL DB (wsprnet, tutorial) and put up a few Scala apps / Py Sample Scripts for CSV conversion testing. I've not added the tutorial tables to the init script yet, but will do soon.

The Docs site is far from complete, but, here are few links so folks can see the direction I'm heading (In time there will be visualization plots). I've added a WSPR-Daemon section where things related to this project will go. The concepts are the same for both WSPRnet and WSPR-Daemon in terms of processing data in volume.

Some examples:

As I said earlier, there's a ton to be added, including a full tools-installation section for those interested in either project using Spark related tooling. I was planning on doing some Vagrant testing of the Daemon script before the holidays ended, but, I did not get to it. There's just to many fun things to do, and not enough time to do them.

Anyway, I look forward to seeing how things progress here with WSPR Daemon !

73's
Greg, KI7MT


Re: Example query and analysis using R

Andi Fugard M0INF
 

Hi Gwyn,

I haven't used Grafana, so I'm not sure.

R has a huge number of packages for pretty much every visualisation and statistical analysis in existence, so there's likely to be something. It's also fairly easy to get from a local analysis to a web app, e.g., here is a simple visualisation grabbing the last hour's worth of sent/receives to/from IO91: https://andif.shinyapps.io/wspr_app/

(It also happens to be software I can use!)

Anyway, shall have a think :)

73,

Andi M0INF


Re: HELP - every day from 14h utc it is the time, when uploads are not accepted any more...

ON5KQ
 

Thanks Rob, for explanation...
I will keep the system running. It was however strange, that wsprnet.org stopped accepting the uploads the last 3 days just after reconfig of the antenna directions (according to wsprdaemon.conf scheduled rearrangement of kiwis)

I will check it the next days, if that keeps making problems at 14h utc...

I checked, but wsprnet.org never accepted spots from 14h to 15h utc at least on 20m... most likely on all bands, but I didn't check the last...
The stored files from that period on 20m disappeared from my PC unfortunately...

Ulli, ON5KQ


Re: Example query and analysis using R

Gwyn Griffiths
 

Hello Andi
I'm glad you have found this group and the WsprDaemon database. 

Thank you for sharing your R example, on how to access, and on how to generate plots.
I will include a link to your web page in our documentation. 

It would be nice to have your views on what visualisations one could program in R that would be useful but are not possible using our Grafana site.

73
Gwyn G3ZIL 


Re: HELP - every day from 14h utc it is the time, when uploads are not accepted any more...

Rob Robinett
 

Sometimes wsprnet.org will report that it accepts only some of the uploaded spots and those spots cannot be uploaded by retrying.
It doesn't tell WD why it rejects spots or what those rejected spot are, so there is nothing more WD can do and I then delete the spot files.
In contrast, uploads to wsprdaeon.org are completely under our control.
To address these problems, I have suggested to Phil that he offer the option to search the extended_spots database rather than the wsprnet scrape.


Example query and analysis using R

Andi Fugard M0INF
 

Greetings,

I just found this database and group - really wonderful. Thanks all for work making it possible.

Here's an example showing how to query the db using R:
https://inductivestep.github.io/WSPR-analysis/

73s,

Andi M0INF


Re: HELP - every day from 14h utc it is the time, when uploads are not accepted any more...

Rob Robinett
 

My Maui WD is uploading to wsprnet.org without problems.
So I infer your problem is unique to your site.
Don't delete any of your _spots.txt, your system has space for months of spots files.
Once WD can connect to wsprnet.org it will upload all of those cached spots in blocks of no more than 1000 spots.
That mechanism worked in Maui and elsewhere during the Jan 1 maintenance outage.
I have resumed work on the WD client and will be adding a configurable reverse proxy facility which will allow users to provide remote access without router changes.


On Sun, Jan 3, 2021 at 8:21 AM ON5KQ <ON5KQ@...> wrote:
It seems to me, that the wsprnet.org website is still not under control ...
The only real problem with this is, that nobody knows, as there is no clear announcements from the system administrators, when there is any maintenance work....

However it COULD be different as well:
- At 14h utc I change antenna configuration here according to the wsprdaemon.conf  definition. It is one of the several times per day where changes are made with the different antennas and Kiwism switching ant directions, etc...
- After WD made the changes as specified in wsprdaemon.conf the wsprnet.org website does not accept spots any more.....no spot uploads at all! 

Instead  in of the folders:
home/wsprdaemon/uploads.d/wsprnet.d/spots.d/ON5KQ_JO10os/kiwi-names/
in each kiwi-name lots of .txt files are written - every 2min a txt file...

To my understanding these are the files, which are waiting for upload and which are NOT uploaded... Naturally they do not appear anywhere on VK7JJs website...

Already now there are 30 files written for my spots decoded locally for the last hour, but not accepted by wsprnet,org....building up, and writing and writing .... no uploads at all!
What should I do ??

In the tmp folder I can check the curl log:
/tmp/wsprdaemon/uploads.d/wsprnet.d/curl.log
It confirms the latest upload was accepted an hour ago.... since then nothing was uploaded...

What should I do ? keep it running ??
Will it be possible to upload the spots later on ? However it will be many many many spots...

Does anyone know, if wsprnet.org is 'partly out of service' ??

Is my spot rate of up to 2500 spots/hour I have seen in the past too much somewhere ? Should I disconnect a few antennas, so spot rate becomes lower ?

Do I need to restart the PC and restart WD?
As I said, it is difficult to know, what to do, if there is no information about, what happens at the server side...

Ulli, ON5KQ

Remark:
Unfortunately my infrastructure here is not able to quickly allow on the fly changes so someone can log-in from outside, as other Internet services have to run permanently.
I therefore cannot reconfigure routers as the routers will not store new settings without complete reboot of the system. Another bug of the hardware I found, as it fails to store settings on the fly...
Time to through out such failing hardware, I know.... I am already in the process to purchase necessary replacements.



--
Rob Robinett
AI6VN
mobile: +1 650 218 8896


Re: HELP - every day from 14h utc it is the time, when uploads are not accepted any more...

ON5KQ
 

After about 70..80min, I see now, that the curl.log shows confirmation of about 1000 spots successful uploaded.

All stored .txt files are erased on my computer.... all should be fine... HOWEVER !

Phil's website (VK7JJ) still shows a gap of 30min with no spots at all.... as I know, usually there are about 2000spots/hour at this time of the day... So 30min gap is about 1000spots missing...
But why the .txt files are all erased from my computer, if wsprnet.org did not acknowledge the upload ??

I thought that after wsprd (the decoder) has generated the spots list on my computer , this list is stored in a txt file and will be only erased, if the curl-upload to wsprnet.org will be successful. If not successful, the .txt file will remain and will be only erased, if a second upload was successful...
Now everything is erased, but still not uploaded ?? What has really happened ? Can I get the missing spots back somewhere to upload them ?

The UN-thinkable:
Does it happen that wsprnet.org acknowledge spot uploads and after that throw away half of them, if the number of uploads is too much for the site ??!  Would be a real serious flaw !

Ulli, ON5KQ


HELP - every day from 14h utc it is the time, when uploads are not accepted any more...

ON5KQ
 

It seems to me, that the wsprnet.org website is still not under control ...
The only real problem with this is, that nobody knows, as there is no clear announcements from the system administrators, when there is any maintenance work....

However it COULD be different as well:
- At 14h utc I change antenna configuration here according to the wsprdaemon.conf  definition. It is one of the several times per day where changes are made with the different antennas and Kiwism switching ant directions, etc...
- After WD made the changes as specified in wsprdaemon.conf the wsprnet.org website does not accept spots any more.....no spot uploads at all! 

Instead  in of the folders:
home/wsprdaemon/uploads.d/wsprnet.d/spots.d/ON5KQ_JO10os/kiwi-names/
in each kiwi-name lots of .txt files are written - every 2min a txt file...

To my understanding these are the files, which are waiting for upload and which are NOT uploaded... Naturally they do not appear anywhere on VK7JJs website...

Already now there are 30 files written for my spots decoded locally for the last hour, but not accepted by wsprnet,org....building up, and writing and writing .... no uploads at all!
What should I do ??

In the tmp folder I can check the curl log:
/tmp/wsprdaemon/uploads.d/wsprnet.d/curl.log
It confirms the latest upload was accepted an hour ago.... since then nothing was uploaded...

What should I do ? keep it running ??
Will it be possible to upload the spots later on ? However it will be many many many spots...

Does anyone know, if wsprnet.org is 'partly out of service' ??

Is my spot rate of up to 2500 spots/hour I have seen in the past too much somewhere ? Should I disconnect a few antennas, so spot rate becomes lower ?

Do I need to restart the PC and restart WD?
As I said, it is difficult to know, what to do, if there is no information about, what happens at the server side...

Ulli, ON5KQ

Remark:
Unfortunately my infrastructure here is not able to quickly allow on the fly changes so someone can log-in from outside, as other Internet services have to run permanently.
I therefore cannot reconfigure routers as the routers will not store new settings without complete reboot of the system. Another bug of the hardware I found, as it fails to store settings on the fly...
Time to through out such failing hardware, I know.... I am already in the process to purchase necessary replacements.


Re: AtomicPi install problem

John
 

Error is cleared.
Thank you Stu.

TI4JWC


Re: AtomicPi install problem

Stu C
 

Try,
sudo apt install python3-numpy

Stu


Re: AtomicPi install problem

John
 

Rob, "APi repo" ?  Is that something you or I need to work on?

John


Re: AtomicPi install problem

Rob Robinett
 

yes, but it should be installed automatically by WD.
Perhaps the APi repo don't have it.

On Fri, Jan 1, 2021 at 2:50 PM John via groups.io <n0ure=yahoo.com@groups.io> wrote:
I finally got a good install of Ubuntu 20.04. I removed all the unnecessary apps that leaves me with 7 G mem.
When installed wsprdaemon I got the error "Package python-numpy has no installation candidate. 

Do I need to install python-numpy ?



--
Rob Robinett
AI6VN
mobile: +1 650 218 8896


AtomicPi install problem

John
 

I finally got a good install of Ubuntu 20.04. I removed all the unnecessary apps that leaves me with 7 G mem.
When installed wsprdaemon I got the error "Package python-numpy has no installation candidate. 

Do I need to install python-numpy ?


Feedback of unusual fault today...

ON5KQ
 

Happy new year to everybody...

As I was focused on family matters today, I didn't pay attention to what happened with the wsprdaemon-receivers. This afternoon however I was astonished of rather less spots than usual reported - especially 40m and up was lacking considerably.
At first I thought some problem with the antenna or feed system occurred, however I found a rather unusual error message I have never seen before.

I checked the file
wsprdaemon/uploads.d/wsprnet.d/spots.d/uploads.log

It showed the following message:
-----------------------------------------------------
tr: write error: Broken pipe
tr: write error: Broken pipe
tr: write error
/home/on5kq/wsprdaemon/wsprdaemon.sh: line 3420: echo: write error: Broken pipe
-----------------------------------------------------


I stopped wsprdaemon with
wsprdaemon.sh -z

Note: before stopping wsprdaemon only the first line was showing in the upload.log file.... The next three lines were added after I stopped wsprdaemon

Then I restarted WD again with
wsprdaemon.sh -a
and as result all works normal again now.

However according to Phil's website there was a 90min gap, where there was no upload at all from ON5KQ
Strange also, that there was nothing stored locally on my PC, not yet uploaded, but waiting for upload.

I want to report it, as it might be important to help the system get more robust (wsprnet.org <--> wsprdaemon.org <--> VK7JJs site and others

I have no idea where this problem came from... may be another glitch at wsprnet.org ??

Ulli, ON5KQ


Re: Ubuntu 20.04 --- no git

John
 

I reinstalled the Ubuntu Full version and all installed. wsprdaemon only needs me to edit the config file.
I will need to clean out all the extra apps and folders to get some space back.
Thanks for the help again.

john

541 - 560 of 668