Kiwi connections not going away


Chris Mackerell
 

Hi

I run a sliding schedule of bands that connect to my KiwiSDRs.

The connections to the Kiwis are not going away at the end
of a scheduled timeframe any more. New connections start all right,
but at the end they don't close completely.

"wd -s" doesn't show the jobs, but the Kiwi admin window still
shows connections. If I "kick" them from there they come
straight back up.

If I shut down wd I get lots of meaningless (to me anyway!)
messages about zombies running around the place.

This used to all work fine, but my Kiwis are now filling
up with unwanted connections at the wrong time of day.
I'm not 100% when this started going wrong, but suspect
it was around when v3 was merged into the master branch.

I've been running "wd -l e" but nothing appears there.

Running on Intel-based Ubuntu.

Any ideas?

Cheers, Chris

--
Chris Mackerell, 217 Sandy Bay-Marahau Road, Marahau
RD 2, Motueka 7197, New Zealand chris@...


Rob Robinett
 

Hi Chris,

Run 'git pull' to get the latest version of 3.0 after which  'wd -V' should print 3.0.2.6.

In beta testing I found that on busy servers the kiwirecord.py process which records the wav files was being interrupted by other programs and as a result there were corrupted wav files.  To address that problem I raised the program priority  of the kiwirecorder jobs, but they have to run as user 'root', so the schedule change must 'sudo kill ...' to them.  With that change those zombie kiwirecorder jobs should be killed.

There is still a lot of log file 'noise' associated with these changes and I am working to clean that up.

73,

Rob


Chris Mackerell
 

Hi Rob

I'm already on 3.0.2.6

It looks like only the WSPR decoding process is getting killed:

wsprdaemon@wsprdaemon:/tmp/wsprdaemon/recording.d/KIWI_1/10$ more decoding_daemon_kill_handler.log
Fri 03 Jun 2022 03:09:07 PM NZST: decoding_daemon_kill_handler() running in /tmp/wsprdaemon/recording.d/KIWI_1/10 with pid 941 i
s processing SIGTERM to stop decoding on KIWI_1,10
Fri 03 Jun 2022 03:09:07 PM NZST: Successful: 'kill_wav_recording_daemon running as pid 941 for KIWI_1 10 => 0

but there is still a kiwirecorder.py running and writing
.wav files to /tmp/wsprdaemon/recording.d/KIWI_1/10

   5464 ?        S      0:00 sudo nice --adjustment=-40 python3 -u /home/wsprdaemon/wsprdaemon/kiwiclient/kiwirecorder.py --freq=28124.6 --server-host=kiwisdr3.owdjim.gen.nz --server-port=8077 --OV --user=wsprdaemon_v3.0.2.6 --password=NULL --agc-gain=60 --quiet --no_compression --modulation=usb --lp-cutoff=1340 --hp-cutoff=1660 --dt-sec=60

Cheers, Chris

On 2022-06-03 16:11, Rob Robinett wrote:
Hi Chris,

Run 'git pull' to get the latest version of 3.0 after which  'wd -V' should print 3.0.2.6.

In beta testing I found that on busy servers the kiwirecord.py process which records the wav files was being interrupted by other programs and as a result there were corrupted wav files.  To address that problem I raised the program priority  of the kiwirecorder jobs, but they have to run as user 'root', so the schedule change must 'sudo kill ...' to them.  With that change those zombie kiwirecorder jobs should be killed.

There is still a lot of log file 'noise' associated with these changes and I am working to clean that up.

73,

Rob


Rob Robinett
 

Is your account set up for your user to 'auto-sudo'?

On Thu, Jun 2, 2022 at 9:29 PM Chris Mackerell <chris@...> wrote:
Hi Rob

I'm already on 3.0.2.6

It looks like only the WSPR decoding process is getting killed:

wsprdaemon@wsprdaemon:/tmp/wsprdaemon/recording.d/KIWI_1/10$ more decoding_daemon_kill_handler.log
Fri 03 Jun 2022 03:09:07 PM NZST: decoding_daemon_kill_handler() running in /tmp/wsprdaemon/recording.d/KIWI_1/10 with pid 941 i
s processing SIGTERM to stop decoding on KIWI_1,10
Fri 03 Jun 2022 03:09:07 PM NZST: Successful: 'kill_wav_recording_daemon running as pid 941 for KIWI_1 10 => 0

but there is still a kiwirecorder.py running and writing
.wav files to /tmp/wsprdaemon/recording.d/KIWI_1/10

   5464 ?        S      0:00 sudo nice --adjustment=-40 python3 -u /home/wsprdaemon/wsprdaemon/kiwiclient/kiwirecorder.py --freq=28124.6 --server-host=kiwisdr3.owdjim.gen.nz --server-port=8077 --OV --user=wsprdaemon_v3.0.2.6 --password=NULL --agc-gain=60 --quiet --no_compression --modulation=usb --lp-cutoff=1340 --hp-cutoff=1660 --dt-sec=60

Cheers, Chris

On 2022-06-03 16:11, Rob Robinett wrote:
Hi Chris,

Run 'git pull' to get the latest version of 3.0 after which  'wd -V' should print 3.0.2.6.

In beta testing I found that on busy servers the kiwirecord.py process which records the wav files was being interrupted by other programs and as a result there were corrupted wav files.  To address that problem I raised the program priority  of the kiwirecorder jobs, but they have to run as user 'root', so the schedule change must 'sudo kill ...' to them.  With that change those zombie kiwirecorder jobs should be killed.

There is still a lot of log file 'noise' associated with these changes and I am working to clean that up.

73,

Rob



--
Rob Robinett
AI6VN
mobile: +1 650 218 8896


Chris Mackerell
 

Yep. At least I have:

wsprdaemon      ALL=(ALL:ALL)   NOPASSWD:ALL

in /etc/sudoers. I assume that's what you mean?

I can login as wsprdaemon and do stuff like "sudo reboot"
without getting prompted for a password anyway.


If I shutdown things with "-z" all the strays get killed off
ok, so I thing that is ok. It's just during "normal operations"
that the kiwirecorders aren't getting killed.

Cheers, Chris


On 2022-06-03 16:31, Rob Robinett wrote:
Is your account set up for your user to 'auto-sudo'?

On Thu, Jun 2, 2022 at 9:29 PM Chris Mackerell <chris@...> wrote:
Hi Rob

I'm already on 3.0.2.6

It looks like only the WSPR decoding process is getting killed:

wsprdaemon@wsprdaemon:/tmp/wsprdaemon/recording.d/KIWI_1/10$ more decoding_daemon_kill_handler.log
Fri 03 Jun 2022 03:09:07 PM NZST: decoding_daemon_kill_handler() running in /tmp/wsprdaemon/recording.d/KIWI_1/10 with pid 941 i
s processing SIGTERM to stop decoding on KIWI_1,10
Fri 03 Jun 2022 03:09:07 PM NZST: Successful: 'kill_wav_recording_daemon running as pid 941 for KIWI_1 10 => 0

but there is still a kiwirecorder.py running and writing
.wav files to /tmp/wsprdaemon/recording.d/KIWI_1/10

   5464 ?        S      0:00 sudo nice --adjustment=-40 python3 -u /home/wsprdaemon/wsprdaemon/kiwiclient/kiwirecorder.py --freq=28124.6 --server-host=kiwisdr3.owdjim.gen.nz --server-port=8077 --OV --user=wsprdaemon_v3.0.2.6 --password=NULL --agc-gain=60 --quiet --no_compression --modulation=usb --lp-cutoff=1340 --hp-cutoff=1660 --dt-sec=60

Cheers, Chris

On 2022-06-03 16:11, Rob Robinett wrote:
Hi Chris,

Run 'git pull' to get the latest version of 3.0 after which  'wd -V' should print 3.0.2.6.

In beta testing I found that on busy servers the kiwirecord.py process which records the wav files was being interrupted by other programs and as a result there were corrupted wav files.  To address that problem I raised the program priority  of the kiwirecorder jobs, but they have to run as user 'root', so the schedule change must 'sudo kill ...' to them.  With that change those zombie kiwirecorder jobs should be killed.

There is still a lot of log file 'noise' associated with these changes and I am working to clean that up.

73,

Rob



--
Rob Robinett
AI6VN
mobile: +1 650 218 8896