performance wsprdaemon on a RPI3
Hey Rob--
OK, I updated to 3.0.3. No worries about the "no kiwi connection"
problem, it eventually sorts itself out.
One thing to note, I'm now running both of my sites on a fresh Ubuntu
20.04 Server (headless) install. WD installation worked well, except
that I needed to manually install libfftw3-3, which is required by the
wsprd binary. The error:
./wsprd: error while loading shared libraries: libfftw3f.so.3: cannot
open shared object file: No such file or directory
Sorry I noticed this a month ago or so, I should have brought it up
sooner! It looks like nobody else has said anything about this, maybe
this package is installed by default on the Desktop GUI versions of 20.04.
Thanks, have a great evening!
--
Bryan Klofas
On 6/13/22 16:45, Rob Robinett wrote:
> Hi Bryan,
>
> I have just published WD 3.0.3 and your site is one of the few beta
> sites I couldn't update. While there are no major changes in the code
> since the 3.0.2.6 version you are running, there are a number of error
> handling enhancements and it would help me support you if you were
> running that 3.0.3 code.
>
> Unfortunately enhancing kiwirecorder error handling as you suggested
> didn't make it into 3.0.3, but I will be working on that part of the
> code and will see what could be done. I encountered several beta sites
> which suffered from the same 'no kiwi connection' problem, and you
> report kept me from panicking while the Kiwis were busy.
>
> 73,
>
> Rob
>
> On Fri, May 27, 2022 at 11:38 AM Bryan Klofas <bklofas@...
> <mailto:bklofas@...>> wrote:
>
> Hey Rob--
>
> Thanks for the tips on decreasing the "-o" flag if I run out of cycles.
> My current KF6ZEO spotting station is actually running on a Ubuntu
> 20.04
> VM with 2 threads of a i3-5010U, so it's got enough horsepower. I'm
> building up a Pi for a remote site.
>
> It might be helpful to add a bit more error logging if kiwirecorder
> can't access the SDR.
>
> For instance, I just stopped wsprdaemon, git pull, started, and viewed
> the error logs. Nothing showed up using "-l e", but no spots were being
> uploaded.
>
> It turns out that as soon as I stopped wsprdaemon, the KiwiSDR started
> to update itself, with no incoming connections allowed. Judging by
> htop,
> it appears that kiwirecorder is crashlooping. Running kiwirecorder by
> hand results in:
>
> kiwi.client.KiwiDownError: 192.168.1.82 <http://192.168.1.82>:
> server is down atm
>
> However, the recovery worked well. As soon as the server came back up,
> kiwirecorder stopped crashing and wsprdaemon started uploading
> spots. So
> maybe I'm getting a bit too into the weeds here.
>
> Thanks for continuing to work on wsprdaemon, have a great weekend!
> --
> Bryan Klofas KF6ZEO
>
> On 5/26/22 18:32, Rob Robinett wrote:
> > Hi Bryan,
> >
> > I see from http://wspr.rocks/livequeries/
> <http://wspr.rocks/livequeries/>
> > <http://wspr.rocks/livequeries/
> <http://wspr.rocks/livequeries/>> that you are still running 3.0.2.3.
> > You would benefit from updating your WD by running:
> >
> > git pull
> > wd -z
> > wd -a
> >
> > I have just installed Wd 3.0.2 at many of the top sites and have
> found
> > how to use some of its features to ensure there is no CPU
> overloading on
> > the WD server. Logging is much improved in WD 3.0, so wsprd
> timeouts
> > and other errors which were silently handled (tom some extent) by WD
> > 2.10 are now much more visible. To see log lines with ERROR execute:
> >
> > wd -l e (show Log errors)
> >
> > You can ignore the "ERROR: only x o y spots accepted by
> wsprnet.org <http://wsprnet.org>"
> > lines, but any other error lines should be investigated.
> >
> > If you see "ERROR: timeout..." lines, then you are running out of
> CPU
> > cycles and you should try adding this line to your conf file:
> >
> > WSPRD_CMD_FLAGS="-C 500 -o 3 -d"
> >
> > At one especially challenged site I had to go down from the
> default '-o
> > 4' to:
> >
> > WSPRD_CMD_FLAGS="-C 500 -o 2 -d"
> >
> > Even '-o 2' didn't seem to affect the number of decoded spots, so
> users
> > with Pi 3bs may be able to scan 14+ bands.
> >
>
>
>
>
>
>
>
> --
> Rob Robinett
> AI6VN
> rob@... <mailto:rob@...>
> mobile: +1 650 218 8896
>
OK, I updated to 3.0.3. No worries about the "no kiwi connection" problem, it eventually sorts itself out.
One thing to note, I'm now running both of my sites on a fresh Ubuntu 20.04 Server (headless) install. WD installation worked well, except that I needed to manually install libfftw3-3, which is required by the wsprd binary. The error:
./wsprd: error while loading shared libraries: libfftw3f.so.3: cannot open shared object file: No such file or directory
Sorry I noticed this a month ago or so, I should have brought it up sooner! It looks like nobody else has said anything about this, maybe this package is installed by default on the Desktop GUI versions of 20.04.
Thanks, have a great evening!
--
Bryan Klofas
Hi Bryan,
I have just published WD 3.0.3 and your site is one of the few beta sites I couldn't update. While there are no major changes in the code since the 3.0.2.6 version you are running, there are a number of error handling enhancements and it would help me support you if you were running that 3.0.3 code.
Unfortunately enhancing kiwirecorder error handling as you suggested didn't make it into 3.0.3, but I will be working on that part of the code and will see what could be done. I encountered several beta sites which suffered from the same 'no kiwi connection' problem, and you report kept me from panicking while the Kiwis were busy.
73,
Rob
On Fri, May 27, 2022 at 11:38 AM Bryan Klofas <bklofas@... <mailto:bklofas@...>> wrote:
Hey Rob--
Thanks for the tips on decreasing the "-o" flag if I run out of cycles.
My current KF6ZEO spotting station is actually running on a Ubuntu
20.04
VM with 2 threads of a i3-5010U, so it's got enough horsepower. I'm
building up a Pi for a remote site.
It might be helpful to add a bit more error logging if kiwirecorder
can't access the SDR.
For instance, I just stopped wsprdaemon, git pull, started, and viewed
the error logs. Nothing showed up using "-l e", but no spots were being
uploaded.
It turns out that as soon as I stopped wsprdaemon, the KiwiSDR started
to update itself, with no incoming connections allowed. Judging by
htop,
it appears that kiwirecorder is crashlooping. Running kiwirecorder by
hand results in:
kiwi.client.KiwiDownError: 192.168.1.82 <http://192.168.1.82>:
server is down atm
However, the recovery worked well. As soon as the server came back up,
kiwirecorder stopped crashing and wsprdaemon started uploading
spots. So
maybe I'm getting a bit too into the weeds here.
Thanks for continuing to work on wsprdaemon, have a great weekend!
--
Bryan Klofas KF6ZEO
On 5/26/22 18:32, Rob Robinett wrote:
> Hi Bryan,
>
> I see from http://wspr.rocks/livequeries/
<http://wspr.rocks/livequeries/>
> <http://wspr.rocks/livequeries/
<http://wspr.rocks/livequeries/>> that you are still running 3.0.2.3.
> You would benefit from updating your WD by running:
>
> git pull
> wd -z
> wd -a
>
> I have just installed Wd 3.0.2 at many of the top sites and have
found
> how to use some of its features to ensure there is no CPU
overloading on
> the WD server. Logging is much improved in WD 3.0, so wsprd
timeouts
> and other errors which were silently handled (tom some extent) by WD
> 2.10 are now much more visible. To see log lines with ERROR execute:
>
> wd -l e (show Log errors)
>
> You can ignore the "ERROR: only x o y spots accepted by
wsprnet.org <http://wsprnet.org>"
> lines, but any other error lines should be investigated.
>
> If you see "ERROR: timeout..." lines, then you are running out of
CPU
> cycles and you should try adding this line to your conf file:
>
> WSPRD_CMD_FLAGS="-C 500 -o 3 -d"
>
> At one especially challenged site I had to go down from the
default '-o
> 4' to:
>
> WSPRD_CMD_FLAGS="-C 500 -o 2 -d"
>
> Even '-o 2' didn't seem to affect the number of decoded spots, so
users
> with Pi 3bs may be able to scan 14+ bands.
>
--
Rob Robinett
AI6VN
rob@... <mailto:rob@...>
mobile: +1 650 218 8896
Hey Rob--
Thanks for the tips on decreasing the "-o" flag if I run out of cycles.
My current KF6ZEO spotting station is actually running on a Ubuntu 20.04
VM with 2 threads of a i3-5010U, so it's got enough horsepower. I'm
building up a Pi for a remote site.
It might be helpful to add a bit more error logging if kiwirecorder
can't access the SDR.
For instance, I just stopped wsprdaemon, git pull, started, and viewed
the error logs. Nothing showed up using "-l e", but no spots were being
uploaded.
It turns out that as soon as I stopped wsprdaemon, the KiwiSDR started
to update itself, with no incoming connections allowed. Judging by htop,
it appears that kiwirecorder is crashlooping. Running kiwirecorder by
hand results in:
kiwi.client.KiwiDownError: 192.168.1.82: server is down atm
However, the recovery worked well. As soon as the server came back up,
kiwirecorder stopped crashing and wsprdaemon started uploading spots. So
maybe I'm getting a bit too into the weeds here.
Thanks for continuing to work on wsprdaemon, have a great weekend!
--
Bryan Klofas KF6ZEO
On 5/26/22 18:32, Rob Robinett wrote:
> Hi Bryan,
>
> I see from http://wspr.rocks/livequeries/
> <http://wspr.rocks/livequeries/> that you are still running 3.0.2.3.
> You would benefit from updating your WD by running:
>
> git pull
> wd -z
> wd -a
>
> I have just installed Wd 3.0.2 at many of the top sites and have found
> how to use some of its features to ensure there is no CPU overloading on
> the WD server. Logging is much improved in WD 3.0, so wsprd timeouts
> and other errors which were silently handled (tom some extent) by WD
> 2.10 are now much more visible. To see log lines with ERROR execute:
>
> wd -l e (show Log errors)
>
> You can ignore the "ERROR: only x o y spots accepted by wsprnet.org"
> lines, but any other error lines should be investigated.
>
> If you see "ERROR: timeout..." lines, then you are running out of CPU
> cycles and you should try adding this line to your conf file:
>
> WSPRD_CMD_FLAGS="-C 500 -o 3 -d"
>
> At one especially challenged site I had to go down from the default '-o
> 4' to:
>
> WSPRD_CMD_FLAGS="-C 500 -o 2 -d"
>
> Even '-o 2' didn't seem to affect the number of decoded spots, so users
> with Pi 3bs may be able to scan 14+ bands.
>
Thanks for the tips on decreasing the "-o" flag if I run out of cycles. My current KF6ZEO spotting station is actually running on a Ubuntu 20.04 VM with 2 threads of a i3-5010U, so it's got enough horsepower. I'm building up a Pi for a remote site.
It might be helpful to add a bit more error logging if kiwirecorder can't access the SDR.
For instance, I just stopped wsprdaemon, git pull, started, and viewed the error logs. Nothing showed up using "-l e", but no spots were being uploaded.
It turns out that as soon as I stopped wsprdaemon, the KiwiSDR started to update itself, with no incoming connections allowed. Judging by htop, it appears that kiwirecorder is crashlooping. Running kiwirecorder by hand results in:
kiwi.client.KiwiDownError: 192.168.1.82: server is down atm
However, the recovery worked well. As soon as the server came back up, kiwirecorder stopped crashing and wsprdaemon started uploading spots. So maybe I'm getting a bit too into the weeds here.
Thanks for continuing to work on wsprdaemon, have a great weekend!
--
Bryan Klofas KF6ZEO
Hi Bryan,
I see from http://wspr.rocks/livequeries/ <http://wspr.rocks/livequeries/> that you are still running 3.0.2.3. You would benefit from updating your WD by running:
git pull
wd -z
wd -a
I have just installed Wd 3.0.2 at many of the top sites and have found how to use some of its features to ensure there is no CPU overloading on the WD server. Logging is much improved in WD 3.0, so wsprd timeouts and other errors which were silently handled (tom some extent) by WD 2.10 are now much more visible. To see log lines with ERROR execute:
wd -l e (show Log errors)
You can ignore the "ERROR: only x o y spots accepted by wsprnet.org" lines, but any other error lines should be investigated.
If you see "ERROR: timeout..." lines, then you are running out of CPU cycles and you should try adding this line to your conf file:
WSPRD_CMD_FLAGS="-C 500 -o 3 -d"
At one especially challenged site I had to go down from the default '-o 4' to:
WSPRD_CMD_FLAGS="-C 500 -o 2 -d"
Even '-o 2' didn't seem to affect the number of decoded spots, so users with Pi 3bs may be able to scan 14+ bands.
I see from http://wspr.rocks/livequeries/ that you are still running 3.0.2.3. You would benefit from updating your WD by running:
git pull
wd -z
wd -a
I have just installed Wd 3.0.2 at many of the top sites and have found how to use some of its features to ensure there is no CPU overloading on the WD server. Logging is much improved in WD 3.0, so wsprd timeouts and other errors which were silently handled (tom some extent) by WD 2.10 are now much more visible. To see log lines with ERROR execute:
wd -l e (show Log errors)
You can ignore the "ERROR: only x o y spots accepted by wsprnet.org" lines, but any other error lines should be investigated.
If you see "ERROR: timeout..." lines, then you are running out of CPU cycles and you should try adding this line to your conf file:
WSPRD_CMD_FLAGS="-C 500 -o 3 -d"
At one especially challenged site I had to go down from the default '-o 4' to:
WSPRD_CMD_FLAGS="-C 500 -o 2 -d"
Even '-o 2' didn't seem to affect the number of decoded spots, so users with Pi 3bs may be able to scan 14+ bands.
I've been staring at "htop" for too long, trying to track down a problem where if wsprd runs for more than 1 minute, spots are not decoded or posted.
I'm running wsprdaemon on a Pi 2 from 40m-10m, which works fine for all bands except 20 meters during the daytime when this problem happens. I know I should upgrade my Pi2, but I can't seem to buy a Pi4....
Upping the verbosity levels of the logs, it appears that many things happen 1 minute after the end of the WSPR period:
Thu Mar 31 00:49:00 UTC 2022: watchdog_daemon() is awake
Thu Mar 31 00:49:00 UTC 2022: running update_master_hashtable()
Thu Mar 31 00:49:01 UTC 2022: update_master_hashtable found no rx/band directories
Thu Mar 31 00:49:01 UTC 2022: spawn_upload_daemons() start
Thu Mar 31 00:49:01 UTC 2022: spawn_upload_to_wsprnet_daemon() INFO: uploading job with pid 12499 is already running
Thu Mar 31 00:49:01 UTC 2022: spawn_ftp_upload_to_wsprdaemon_daemon() INFO: uploading job for '/home/pi/wsprdaemon-bklofas/uploads.d/wsprdaemon.d' with pid 12502 is already running
.......
Thu Mar 31 00:49:02 UTC 2022: check_for_zombies() Found running_pid '12478' in expected_pids ' 12478 12499 12502 12565 12609 12699 12696'
Thu Mar 31 00:49:02 UTC 2022: check_for_zombies() Found running_pid '12499' in expected_pids ' 12478 12499 12502 12565 12609 12699 12696'
Thu Mar 31 00:49:02 UTC 2022: check_for_zombies() Found running_pid '12502' in expected_pids ' 12478 12499 12502 12565 12609 12699 12696'
Thu Mar 31 00:49:02 UTC 2022: check_for_zombies() Found running_pid '12565' in expected_pids ' 12478 12499 12502 12565 12609 12699 12696'
Thu Mar 31 00:49:02 UTC 2022: check_for_zombies() Found running_pid '12609' in expected_pids ' 12478 12499 12502 12565 12609 12699 12696'
Thu Mar 31 00:49:02 UTC 2022: check_for_zombies() Found running_pid '12696' in expected_pids ' 12478 12499 12502 12565 12609 12699 12696'
Thu Mar 31 00:49:02 UTC 2022: check_for_zombies() Found running_pid '12699' in expected_pids ' 12478 12499 12502 12565 12609 12699 12696'
Thu Mar 31 00:49:02 UTC 2022: check_for_zombies() WARNING: did not find running_pid '3207' in expected_pids ' 12478 12499 12502 12565 12609 12699 12696'
pi 3207 0.0 0.0 6728 456 pts/1 S 00:48 0:00 \_ timeout 110 nice /home/pi/wsprdaemon-bklofas/bin/wsprd -c -C 500 -o 4 -d -w -f 14.0956 220331_0046.wav
Thu Mar 31 00:49:02 UTC 2022: check_for_zombies() adding running zombie '3207' to kill list
Thu Mar 31 00:49:02 UTC 2022: check_for_zombies() WARNING: did not find running_pid '3208' in expected_pids ' 12478 12499 12502 12565 12609 12699 12696'
pi 3208 100 1.3 14944 13148 pts/1 RN 00:48 0:52 \_ /home/pi/wsprdaemon-bklofas/bin/wsprd -c -C 500 -o 4 -d -w -f 14.0956 220331_0046.wav
Thu Mar 31 00:49:02 UTC 2022: check_for_zombies() adding running zombie '3208' to kill list
Thu Mar 31 00:49:02 UTC 2022: check_for_zombies() killing pids ' 3207 3208'
PID TTY STAT TIME COMMAND
3207 pts/1 S 0:00 timeout 110 nice /home/pi/wsprdaemon-bklofas/bin/wsprd -c -C 500 -o 4 -d -w -f 14.0956 220331_0046.wav
3208 pts/1 RN 0:52 /home/pi/wsprdaemon-bklofas/bin/wsprd -c -C 500 -o 4 -d -w -f 14.0956 220331_0046.wav
So from what I can tell, the WSPR period is 00:46 to 00:48, then wsprd gets called at 00:48:06. So far so good. But then at 00:49:01, it tries to upload the decoded spots for this period. But wsprd hasn't finished, so there are no spots to upload.
Then at 00:49:02 zombies are checked for, and the running wsprd process is found and killed.
Is my analysis of the logs correct? So even though wsprd is called with "timeout 110" and nice, it actually needs to finish within 52 real seconds (because wsprd gets started 6 seconds after the WSPR period ends). Is it possible to push this 1 minute to 100 seconds or a bit longer?
I'm not sure which is the bigger issue: uploading spots before wsprd is finished, or killing wsprd. Allowing wsprd to go for 100 seconds wouldn't matter if the spot uploader checks at 52 seconds.
Thanks!
--
Bryan Klofas KF6ZEO
Hi Hans,
You are of course welcome to experiment with increasing the wsprd TIMEOUT value beyond the default beyond 110 seconds, but if it takes wsprd longer than 120 seconds to execute, then the WD decoding thread will gradually fall behind the wav file recording and those wav files will fill up your 3000 MBtye //tmp/wsprdaemon file system to overflowing. Your reports stimulated me to study the performance of the WD Pi at KPH where there are 14 bands of WD decoding running on a Pi 4. I found that in order for that Pi to finish its wsprd withing 120 seconds I had to modify the config file by adding the line:
WSPRD_CMD_FLAGS="-C 500 -o 3 -d"
You can't use the summary lines in 'top' without understanding that there are 4 cpu cores in the Pi's ARM CPU and the wsprd of each band runs on only one of those 4 cores. So if your 40M wspd were the only code running for the whole of the 120 second wsprd cycle, 'top' would show the CPU as only 25% busy. Type '1' while in 'top' and it will show you the state of each of the CPU cores.
But you are fortunately to have a good receive site with so many signals coming from your antenna that your QTH deserves a i5 rather than cripple 'wsprd' in order to get it to run on a Pi.
73,
Rob
On Wed, Mar 2, 2022 at 4:20 AM Hans V alphen via groups.io <http://groups.io> <pa0ehg@... <mailto:amsat.org@groups.io>> wrote:
Hi Rob,
I have been running WSPRdaemon on a RPI3 for a few days now and I
have many problems looking as overload of the RPI.
Looking at the processor from the RPI I see a CPU usage which never
exceeds 90%, I am only using the RPI for two frequencies WSPR on
80eu and 80 mtr.
It looks like the CPU usage is not the big problem I find. I have
the impression it has to do with a time out of 110 seconds. I have
been looking into wsprdaemon.sh and I do find timeout but to be
honest I do not quite understand what all is done in that part of
the wsprdaemon.sh.
What I see in practical performance is a sudden breakdown in
performance. I think It's at the point where the time out is
exceeded. As soon as this happens the WSPRDaemon is no longer
decoding spots. It looks like .wav files are missing and decoding is
not possible.
It seems to happen as soon as evening conditions start and it
continues until the end of evening conditions. During the night
there are sometimes period with no problems but as soon as I get 2
minute intervalls with close to 10 decodable spots (I know this from
my other setup) than it collapses.
I have two pictures where you can see what is happening.
The first picture shows the analysis of the Kiwi with the RPI3, as
soon as evening conditions start the spots per over begin to hevave
strange. The second picture is the analysis of my second setup, it's
using the same antenna as the Kiwi and as you can see performance is
stable and much more spots decoded. Mind that the two pictures have
a different spots per over interval in the diagram.
Looking at the noise measurements I see a Kiwi which in my opinion
is working OK, I have carefully adjusted the input level so that
during the night it does not overload, which it did at my first few
days. Also the CPU usage of the KIwi is not very high and it does
not look like reason of the performance degradation.
Now my question: would it be worth to try and increase the time out
period of 110 seconds and how could this be done ??
Best solution probably is leaving the RPI3 for what it is and use a
better machine. This morning I received a refurbished I5 for that
but I have to see and find to get Ubuntu on it and get things going.
So perhaps your advice could also be to leave the RPI3 and not
experiment with the time out. On the other hand as the CPU usage of
the RPI3 does not look to be the big problem it would be interesting
to try and understand the problem,.
73's Hans DL/PA0EHG
--
Rob Robinett
AI6VN
rob@... <mailto:rob@...>
mobile: +1 650 218 8896
Hi Erwin,
Thanks for your suggestion. I had already found this possibility and also tried to use this solution.
It seemed to install on the Windows system but if I click on Ubuntu I get a error that my Machine is not correctly installed and that I should allow a virtual machine on Windows. Then I searched how to do this and tried tomake the setting to allow a virtual machine on Windows but as I found this setting it was already enabled. Dead end nr 2 !!!!!
73's Hans DL/PA0EHG
On Wed, Mar 9, 2022 at 01:05 AM, Erwin - PE3ES - F4VTQ wrote:
Hans, and others,
There might be another way to get Ubuntu play nice with Windows, not as a dual boot machine next to Windows so you can only run 1 at a time.
I started experimenting with the Linux subsystem for Windows. That is an extra feature of newer Windows version. You switch it on, install Ubuntu from the Windows Store on it/in it and it runs happily inside the Windows machine and not as a Virtual Machine but as a program (WINE for Windows instead of for Linux).
wiki/Windows_Subsystem_for_Linux
I want it to run my de-dupe bash script on "Windows software generated ALL_WSPR.TXT files" before doing the database upload.
I already installed wsprdaemon 2.10k on it and works brilliant with the KiwiSDR !
There might be another way to get Ubuntu play nice with Windows, not as a dual boot machine next to Windows so you can only run 1 at a time.
I started experimenting with the Linux subsystem for Windows. That is an extra feature of newer Windows version. You switch it on, install Ubuntu from the Windows Store on it/in it and it runs happily inside the Windows machine and not as a Virtual Machine but as a program (WINE for Windows instead of for Linux).
wiki/Windows_Subsystem_for_Linux
I want it to run my de-dupe bash script on "Windows software generated ALL_WSPR.TXT files" before doing the database upload.
I already installed wsprdaemon 2.10k on it and works brilliant with the KiwiSDR !
Unique decodes or phantom decodes are the result of the decoding proces of WSPR. The proces is not that strong and therefore phantom decodes exist. I don't think that it has to do with the decoding software but it is a result on how WSPR is designed and using ways of error detection.
All WSPR decoders, I think, can output phantom decodes.
Not sure what causes these Phantom decodes, most probably it's an interference by other stations on almost exact the same frequency.
Looking at the WSPR challenge score you see that many reporting stations produce unique decodes. Over a longer period I guess there are only a very few reporters who never have an error log with unique decodes.
73's Hans DL/PA0EHG
On Tue, Mar 8, 2022 at 08:57 AM, WA2TP - Tom wrote:
Hans,your 80 score looks very good.You did however have a few unique decodes (decode that no one else heard, often phantom decodes) which you can see here in the error log.I do not know the reason form these anomalies but they do happen often, to many people.TomWA2TP
From: wsprdaemon@groups.io <wsprdaemon@groups.io> on behalf of Hans V alphen via groups.io <pa0ehg@...>
Sent: Tuesday, March 8, 2022 11:29 AM
To: wsprdaemon@groups.io <wsprdaemon@groups.io>
Subject: Re: [wsprdaemon] performance wsprdaemon on a RPI3Hello Rob,
Just wanted to let you know that I added the line you suggested to my config file. It worked out great, the RPI3 is working fine now. I have it running at 4 different frequencies without big problems. When I use only 3 frequencies I don't have any problems at all.
So your suggestion was excellent and I am happy with it.My Kiwi and the RPI3 got me a very good 80 mtr topscore yesterday.
The Kiwi is about 2% better performing than my Red Pitaya, The Kiwi is about 5 % less than my Perseus and WSJT-X in deep search.
So performance is good and I will start experimenting with merging receivers shortly.
I still am fighting the Ubuntu installation next to my Windows 10 on the I5 computer. It's installed but I cannot select Ubunta at startup. It will take some more time before that's done, Hi.
Best 73's Hans DL/PA0EHG
On Wed, Mar 2, 2022 at 06:59 AM, Rob Robinett wrote:Hi Hans,You are of course welcome to experiment with increasing the wsprd TIMEOUT value beyond the default beyond 110 seconds, but if it takes wsprd longer than 120 seconds to execute, then the WD decoding thread will gradually fall behind the wav file recording and those wav files will fill up your 3000 MBtye //tmp/wsprdaemon file system to overflowing. Your reports stimulated me to study the performance of the WD Pi at KPH where there are 14 bands of WD decoding running on a Pi 4. I found that in order for that Pi to finish its wsprd withing 120 seconds I had to modify the config file by adding the line:
WSPRD_CMD_FLAGS="-C 500 -o 3 -d"You can't use the summary lines in 'top' without understanding that there are 4 cpu cores in the Pi's ARM CPU and the wsprd of each band runs on only one of those 4 cores. So if your 40M wspd were the only code running for the whole of the 120 second wsprd cycle, 'top' would show the CPU as only 25% busy. Type '1' while in 'top' and it will show you the state of each of the CPU cores.But you are fortunately to have a good receive site with so many signals coming from your antenna that your QTH deserves a i5 rather than cripple 'wsprd' in order to get it to run on a Pi.73,Rob
On Wed, Mar 2, 2022 at 4:20 AM Hans V alphen via groups.io <pa0ehg=amsat.org@groups.io> wrote:Hi Rob,
I have been running WSPRdaemon on a RPI3 for a few days now and I have many problems looking as overload of the RPI.
Looking at the processor from the RPI I see a CPU usage which never exceeds 90%, I am only using the RPI for two frequencies WSPR on 80eu and 80 mtr.It looks like the CPU usage is not the big problem I find. I have the impression it has to do with a time out of 110 seconds. I have been looking into wsprdaemon.sh and I do find timeout but to be honest I do not quite understand what all is done in that part of the wsprdaemon.sh.
What I see in practical performance is a sudden breakdown in performance. I think It's at the point where the time out is exceeded. As soon as this happens the WSPRDaemon is no longer decoding spots. It looks like .wav files are missing and decoding is not possible.It seems to happen as soon as evening conditions start and it continues until the end of evening conditions. During the night there are sometimes period with no problems but as soon as I get 2 minute intervalls with close to 10 decodable spots (I know this from my other setup) than it collapses.
I have two pictures where you can see what is happening.
The first picture shows the analysis of the Kiwi with the RPI3, as soon as evening conditions start the spots per over begin to hevave strange. The second picture is the analysis of my second setup, it's using the same antenna as the Kiwi and as you can see performance is stable and much more spots decoded. Mind that the two pictures have a different spots per over interval in the diagram.
Looking at the noise measurements I see a Kiwi which in my opinion is working OK, I have carefully adjusted the input level so that during the night it does not overload, which it did at my first few days. Also the CPU usage of the KIwi is not very high and it does not look like reason of the performance degradation.
Now my question: would it be worth to try and increase the time out period of 110 seconds and how could this be done ??
Best solution probably is leaving the RPI3 for what it is and use a better machine. This morning I received a refurbished I5 for that but I have to see and find to get Ubuntu on it and get things going. So perhaps your advice could also be to leave the RPI3 and not experiment with the time out. On the other hand as the CPU usage of the RPI3 does not look to be the big problem it would be interesting to try and understand the problem,.
73's Hans DL/PA0EHG
--
Sent: Tuesday, March 8, 2022 11:29 AM
To: wsprdaemon@groups.io <wsprdaemon@groups.io>
Subject: Re: [wsprdaemon] performance wsprdaemon on a RPI3
Hello Rob,
Just wanted to let you know that I added the line you suggested to my config file. It worked out great, the RPI3 is working fine now. I have it running at 4 different frequencies without big problems. When I use only 3 frequencies I don't have any problems
at all.
So your suggestion was excellent and I am happy with it.
My Kiwi and the RPI3 got me a very good 80 mtr topscore yesterday.
The Kiwi is about 2% better performing than my Red Pitaya, The Kiwi is about 5 % less than my Perseus and WSJT-X in deep search.
So performance is good and I will start experimenting with merging receivers shortly.
I still am fighting the Ubuntu installation next to my Windows 10 on the I5 computer. It's installed but I cannot select Ubunta at startup. It will take some more time before that's done, Hi.
Best 73's Hans DL/PA0EHG
On Wed, Mar 2, 2022 at 06:59 AM, Rob Robinett wrote:
Hi Hans,You are of course welcome to experiment with increasing the wsprd TIMEOUT value beyond the default beyond 110 seconds, but if it takes wsprd longer than 120 seconds to execute, then the WD decoding thread will gradually fall behind the wav file recording and those wav files will fill up your 3000 MBtye //tmp/wsprdaemon file system to overflowing. Your reports stimulated me to study the performance of the WD Pi at KPH where there are 14 bands of WD decoding running on a Pi 4. I found that in order for that Pi to finish its wsprd withing 120 seconds I had to modify the config file by adding the line:
WSPRD_CMD_FLAGS="-C 500 -o 3 -d"You can't use the summary lines in 'top' without understanding that there are 4 cpu cores in the Pi's ARM CPU and the wsprd of each band runs on only one of those 4 cores. So if your 40M wspd were the only code running for the whole of the 120 second wsprd cycle, 'top' would show the CPU as only 25% busy. Type '1' while in 'top' and it will show you the state of each of the CPU cores.But you are fortunately to have a good receive site with so many signals coming from your antenna that your QTH deserves a i5 rather than cripple 'wsprd' in order to get it to run on a Pi.73,Rob
On Wed, Mar 2, 2022 at 4:20 AM Hans V alphen via groups.io <pa0ehg=amsat.org@groups.io> wrote:Hi Rob,
I have been running WSPRdaemon on a RPI3 for a few days now and I have many problems looking as overload of the RPI.
Looking at the processor from the RPI I see a CPU usage which never exceeds 90%, I am only using the RPI for two frequencies WSPR on 80eu and 80 mtr.It looks like the CPU usage is not the big problem I find. I have the impression it has to do with a time out of 110 seconds. I have been looking into wsprdaemon.sh and I do find timeout but to be honest I do not quite understand what all is done in that part of the wsprdaemon.sh.
What I see in practical performance is a sudden breakdown in performance. I think It's at the point where the time out is exceeded. As soon as this happens the WSPRDaemon is no longer decoding spots. It looks like .wav files are missing and decoding is not possible.It seems to happen as soon as evening conditions start and it continues until the end of evening conditions. During the night there are sometimes period with no problems but as soon as I get 2 minute intervalls with close to 10 decodable spots (I know this from my other setup) than it collapses.
I have two pictures where you can see what is happening.
The first picture shows the analysis of the Kiwi with the RPI3, as soon as evening conditions start the spots per over begin to hevave strange. The second picture is the analysis of my second setup, it's using the same antenna as the Kiwi and as you can see performance is stable and much more spots decoded. Mind that the two pictures have a different spots per over interval in the diagram.
Looking at the noise measurements I see a Kiwi which in my opinion is working OK, I have carefully adjusted the input level so that during the night it does not overload, which it did at my first few days. Also the CPU usage of the KIwi is not very high and it does not look like reason of the performance degradation.
Now my question: would it be worth to try and increase the time out period of 110 seconds and how could this be done ??
Best solution probably is leaving the RPI3 for what it is and use a better machine. This morning I received a refurbished I5 for that but I have to see and find to get Ubuntu on it and get things going. So perhaps your advice could also be to leave the RPI3 and not experiment with the time out. On the other hand as the CPU usage of the RPI3 does not look to be the big problem it would be interesting to try and understand the problem,.
73's Hans DL/PA0EHG
--
Hello Rob,
Just wanted to let you know that I added the line you suggested to my config file. It worked out great, the RPI3 is working fine now. I have it running at 4 different frequencies without big problems. When I use only 3 frequencies I don't have any problems at all.
So your suggestion was excellent and I am happy with it.
My Kiwi and the RPI3 got me a very good 80 mtr topscore yesterday.
The Kiwi is about 2% better performing than my Red Pitaya, The Kiwi is about 5 % less than my Perseus and WSJT-X in deep search.
So performance is good and I will start experimenting with merging receivers shortly.
I still am fighting the Ubuntu installation next to my Windows 10 on the I5 computer. It's installed but I cannot select Ubunta at startup. It will take some more time before that's done, Hi.
Best 73's Hans DL/PA0EHG
On Wed, Mar 2, 2022 at 06:59 AM, Rob Robinett wrote:
Hi Hans,You are of course welcome to experiment with increasing the wsprd TIMEOUT value beyond the default beyond 110 seconds, but if it takes wsprd longer than 120 seconds to execute, then the WD decoding thread will gradually fall behind the wav file recording and those wav files will fill up your 3000 MBtye //tmp/wsprdaemon file system to overflowing. Your reports stimulated me to study the performance of the WD Pi at KPH where there are 14 bands of WD decoding running on a Pi 4. I found that in order for that Pi to finish its wsprd withing 120 seconds I had to modify the config file by adding the line:
WSPRD_CMD_FLAGS="-C 500 -o 3 -d"You can't use the summary lines in 'top' without understanding that there are 4 cpu cores in the Pi's ARM CPU and the wsprd of each band runs on only one of those 4 cores. So if your 40M wspd were the only code running for the whole of the 120 second wsprd cycle, 'top' would show the CPU as only 25% busy. Type '1' while in 'top' and it will show you the state of each of the CPU cores.But you are fortunately to have a good receive site with so many signals coming from your antenna that your QTH deserves a i5 rather than cripple 'wsprd' in order to get it to run on a Pi.73,Rob
On Wed, Mar 2, 2022 at 4:20 AM Hans V alphen via groups.io <pa0ehg=amsat.org@groups.io> wrote:Hi Rob,
I have been running WSPRdaemon on a RPI3 for a few days now and I have many problems looking as overload of the RPI.
Looking at the processor from the RPI I see a CPU usage which never exceeds 90%, I am only using the RPI for two frequencies WSPR on 80eu and 80 mtr.It looks like the CPU usage is not the big problem I find. I have the impression it has to do with a time out of 110 seconds. I have been looking into wsprdaemon.sh and I do find timeout but to be honest I do not quite understand what all is done in that part of the wsprdaemon.sh.
What I see in practical performance is a sudden breakdown in performance. I think It's at the point where the time out is exceeded. As soon as this happens the WSPRDaemon is no longer decoding spots. It looks like .wav files are missing and decoding is not possible.It seems to happen as soon as evening conditions start and it continues until the end of evening conditions. During the night there are sometimes period with no problems but as soon as I get 2 minute intervalls with close to 10 decodable spots (I know this from my other setup) than it collapses.
I have two pictures where you can see what is happening.
The first picture shows the analysis of the Kiwi with the RPI3, as soon as evening conditions start the spots per over begin to hevave strange. The second picture is the analysis of my second setup, it's using the same antenna as the Kiwi and as you can see performance is stable and much more spots decoded. Mind that the two pictures have a different spots per over interval in the diagram.
Looking at the noise measurements I see a Kiwi which in my opinion is working OK, I have carefully adjusted the input level so that during the night it does not overload, which it did at my first few days. Also the CPU usage of the KIwi is not very high and it does not look like reason of the performance degradation.
Now my question: would it be worth to try and increase the time out period of 110 seconds and how could this be done ??
Best solution probably is leaving the RPI3 for what it is and use a better machine. This morning I received a refurbished I5 for that but I have to see and find to get Ubuntu on it and get things going. So perhaps your advice could also be to leave the RPI3 and not experiment with the time out. On the other hand as the CPU usage of the RPI3 does not look to be the big problem it would be interesting to try and understand the problem,.
73's Hans DL/PA0EHG
--
WSPRD_CMD_FLAGS="-C 500 -o 3 -d"
Hi Rob,
I have been running WSPRdaemon on a RPI3 for a few days now and I have many problems looking as overload of the RPI.
Looking at the processor from the RPI I see a CPU usage which never exceeds 90%, I am only using the RPI for two frequencies WSPR on 80eu and 80 mtr.It looks like the CPU usage is not the big problem I find. I have the impression it has to do with a time out of 110 seconds. I have been looking into wsprdaemon.sh and I do find timeout but to be honest I do not quite understand what all is done in that part of the wsprdaemon.sh.
What I see in practical performance is a sudden breakdown in performance. I think It's at the point where the time out is exceeded. As soon as this happens the WSPRDaemon is no longer decoding spots. It looks like .wav files are missing and decoding is not possible.It seems to happen as soon as evening conditions start and it continues until the end of evening conditions. During the night there are sometimes period with no problems but as soon as I get 2 minute intervalls with close to 10 decodable spots (I know this from my other setup) than it collapses.
I have two pictures where you can see what is happening.
The first picture shows the analysis of the Kiwi with the RPI3, as soon as evening conditions start the spots per over begin to hevave strange. The second picture is the analysis of my second setup, it's using the same antenna as the Kiwi and as you can see performance is stable and much more spots decoded. Mind that the two pictures have a different spots per over interval in the diagram.
Looking at the noise measurements I see a Kiwi which in my opinion is working OK, I have carefully adjusted the input level so that during the night it does not overload, which it did at my first few days. Also the CPU usage of the KIwi is not very high and it does not look like reason of the performance degradation.
Now my question: would it be worth to try and increase the time out period of 110 seconds and how could this be done ??
Best solution probably is leaving the RPI3 for what it is and use a better machine. This morning I received a refurbished I5 for that but I have to see and find to get Ubuntu on it and get things going. So perhaps your advice could also be to leave the RPI3 and not experiment with the time out. On the other hand as the CPU usage of the RPI3 does not look to be the big problem it would be interesting to try and understand the problem,.
73's Hans DL/PA0EHG
Hi Rob,
I have been running WSPRdaemon on a RPI3 for a few days now and I have many problems looking as overload of the RPI.
Looking at the processor from the RPI I see a CPU usage which never exceeds 90%, I am only using the RPI for two frequencies WSPR on 80eu and 80 mtr.
It looks like the CPU usage is not the big problem I find. I have the impression it has to do with a time out of 110 seconds. I have been looking into wsprdaemon.sh and I do find timeout but to be honest I do not quite understand what all is done in that part of the wsprdaemon.sh.
What I see in practical performance is a sudden breakdown in performance. I think It's at the point where the time out is exceeded. As soon as this happens the WSPRDaemon is no longer decoding spots. It looks like .wav files are missing and decoding is not possible.
It seems to happen as soon as evening conditions start and it continues until the end of evening conditions. During the night there are sometimes period with no problems but as soon as I get 2 minute intervalls with close to 10 decodable spots (I know this from my other setup) than it collapses.
I have two pictures where you can see what is happening.
The first picture shows the analysis of the Kiwi with the RPI3, as soon as evening conditions start the spots per over begin to hevave strange. The second picture is the analysis of my second setup, it's using the same antenna as the Kiwi and as you can see performance is stable and much more spots decoded. Mind that the two pictures have a different spots per over interval in the diagram.
Looking at the noise measurements I see a Kiwi which in my opinion is working OK, I have carefully adjusted the input level so that during the night it does not overload, which it did at my first few days. Also the CPU usage of the KIwi is not very high and it does not look like reason of the performance degradation.
Now my question: would it be worth to try and increase the time out period of 110 seconds and how could this be done ??
Best solution probably is leaving the RPI3 for what it is and use a better machine. This morning I received a refurbished I5 for that but I have to see and find to get Ubuntu on it and get things going. So perhaps your advice could also be to leave the RPI3 and not experiment with the time out. On the other hand as the CPU usage of the RPI3 does not look to be the big problem it would be interesting to try and understand the problem,.
73's Hans DL/PA0EHG
