wsprdaemon tmp file questions


Rob Robinett
 

Hi Chris,

How nice to hear from you.  I have finally resumed work on WD version 3.0 after being interrupted by a house move last February.

LX1DQ experienced the same problem as you, and your problem is probably the same as his: SIGNAL_LEVEL_UPLOAD_ID cannot contain a '/'

So change your SIGNAL_LEVEL_UPLOAD_ID to "K8NVH_X" and you should experience no file system overflows.

If you also add:  SIGNAL_LEVEL_UPLOAD="yes"  you will be able to use the excellent Grafana tools at wspr.live to disp your noise ands spots history and much more.

I have cc'd this to the wsprdemon group so others who encounter it might find this fix.

73,

Rob

On Tue, Sep 7, 2021 at 11:41 AM Chris M <nvhengr@...> wrote:
Hi Rob,

It has been a long time since I last wrote with an issue, I hope all is going well and you are not yet tired of hearing from me :-)  

Some months ago, my wsprdaemon stopped uploading.  I had just installed a new firewall and it took some days for my network to stabilize, so I did not try to fix it quickly.  Today I finally dealt with it.  wsprdaemon is again running and presumably uploading things.

However...

I found the problem was not internet access but, rather, the file system was full.  I removed ./uploads.d/wsprdaemon.d/noise.d and that provided enough space to git pull the latest version and then wsprdaemon -a to make it all run.

But the directories ./uploads.d/wsprdaemon.d/noise.d, ./uploads.d/wsprdaemon.d/spots.d, and ./uploads.d/wsprnet.d/spots.d are growing [albeit slowly], not shrinking.  So whatever the problem was, it is still happening.  Any suggestions  as to where I should look to fix this?

Here is the output from du, when run in the ~/wsprdaemon directory:

4       ./.git/branches
8       ./.git/refs/tags
8       ./.git/refs/heads
16      ./.git/refs/remotes/origin
20      ./.git/refs/remotes
40      ./.git/refs
16      ./.git/objects/46
12      ./.git/objects/1b
12      ./.git/objects/ee
12      ./.git/objects/e6
28      ./.git/objects/50
8       ./.git/objects/39
8       ./.git/objects/29
112     ./.git/objects/60
8       ./.git/objects/54
100     ./.git/objects/af
8       ./.git/objects/4a
12      ./.git/objects/37
8       ./.git/objects/41
16      ./.git/objects/f7
28      ./.git/objects/89
8       ./.git/objects/cc
8       ./.git/objects/25
76      ./.git/objects/7b
8       ./.git/objects/a3
108     ./.git/objects/bd
56      ./.git/objects/c8
116     ./.git/objects/f4
112     ./.git/objects/b6
12      ./.git/objects/63
12      ./.git/objects/70
8       ./.git/objects/3e
8       ./.git/objects/65
8       ./.git/objects/fa
16      ./.git/objects/45
28      ./.git/objects/aa
8       ./.git/objects/ac
8       ./.git/objects/cd
12      ./.git/objects/19
8       ./.git/objects/d0
16      ./.git/objects/9f
12      ./.git/objects/01
8       ./.git/objects/68
8       ./.git/objects/6a
12      ./.git/objects/bf
8       ./.git/objects/00
12      ./.git/objects/b2
8       ./.git/objects/7a
12      ./.git/objects/e0
16      ./.git/objects/99
16      ./.git/objects/d8
8       ./.git/objects/14
20      ./.git/objects/04
104     ./.git/objects/d1
8       ./.git/objects/0f
8       ./.git/objects/10
104     ./.git/objects/da
8       ./.git/objects/16
12      ./.git/objects/f5
8       ./.git/objects/ed
20      ./.git/objects/2a
24      ./.git/objects/21
12      ./.git/objects/c6
8       ./.git/objects/fe
16      ./.git/objects/76
8       ./.git/objects/a9
8       ./.git/objects/62
8       ./.git/objects/61
16      ./.git/objects/f8
8       ./.git/objects/e5
8       ./.git/objects/03
8       ./.git/objects/ff
12      ./.git/objects/dc
8       ./.git/objects/87
8       ./.git/objects/f6
8       ./.git/objects/84
112     ./.git/objects/5c
8       ./.git/objects/49
16      ./.git/objects/b0
104     ./.git/objects/0b
104     ./.git/objects/a2
12      ./.git/objects/20
72      ./.git/objects/98
16      ./.git/objects/85
8       ./.git/objects/ce
8       ./.git/objects/6b
12      ./.git/objects/33
8       ./.git/objects/52
580     ./.git/objects/pack
8       ./.git/objects/d2
16      ./.git/objects/34
16      ./.git/objects/ca
8       ./.git/objects/13
20      ./.git/objects/3f
116     ./.git/objects/d5
12      ./.git/objects/9c
8       ./.git/objects/e2
12      ./.git/objects/64
12      ./.git/objects/c1
108     ./.git/objects/c2
24      ./.git/objects/5b
8       ./.git/objects/4f
8       ./.git/objects/15
20      ./.git/objects/3d
20      ./.git/objects/08
8       ./.git/objects/d9
108     ./.git/objects/5a
124     ./.git/objects/de
104     ./.git/objects/97
112     ./.git/objects/c4
20      ./.git/objects/c9
12      ./.git/objects/b4
8       ./.git/objects/ec
8       ./.git/objects/22
72      ./.git/objects/55
32      ./.git/objects/0d
112     ./.git/objects/c7
36      ./.git/objects/36
104     ./.git/objects/72
20      ./.git/objects/a1
8       ./.git/objects/b1
8       ./.git/objects/74
12      ./.git/objects/4d
108     ./.git/objects/b3
72      ./.git/objects/3b
16      ./.git/objects/8d
36      ./.git/objects/86
20      ./.git/objects/2c
12      ./.git/objects/cb
16      ./.git/objects/f9
12      ./.git/objects/1e
12      ./.git/objects/0e
104     ./.git/objects/81
108     ./.git/objects/67
16      ./.git/objects/95
8       ./.git/objects/7d
8       ./.git/objects/fd
8       ./.git/objects/69
16      ./.git/objects/2d
12      ./.git/objects/f2
108     ./.git/objects/c0
12      ./.git/objects/1d
20      ./.git/objects/0c
8       ./.git/objects/8b
8       ./.git/objects/6f
8       ./.git/objects/e3
16      ./.git/objects/b7
16      ./.git/objects/07
108     ./.git/objects/d3
8       ./.git/objects/a0
12      ./.git/objects/dd
104     ./.git/objects/a5
16      ./.git/objects/db
8       ./.git/objects/78
8       ./.git/objects/e9
20      ./.git/objects/a4
8       ./.git/objects/23
20      ./.git/objects/ef
4       ./.git/objects/info
12      ./.git/objects/b9
24      ./.git/objects/8e
8       ./.git/objects/7c
8       ./.git/objects/a7
72      ./.git/objects/3a
8       ./.git/objects/96
8       ./.git/objects/35
8       ./.git/objects/5e
20      ./.git/objects/57
12      ./.git/objects/42
8       ./.git/objects/06
20      ./.git/objects/ba
120     ./.git/objects/6e
12      ./.git/objects/2e
8       ./.git/objects/f0
104     ./.git/objects/9e
8       ./.git/objects/3c
16      ./.git/objects/58
5556    ./.git/objects
8       ./.git/logs/refs/heads
16      ./.git/logs/refs/remotes/origin
20      ./.git/logs/refs/remotes
32      ./.git/logs/refs
40      ./.git/logs
48      ./.git/hooks
8       ./.git/info
5728    ./.git
1688    ./uploads.d/wsprnet.d/spots.d/K8NVH=X_EN82lm/KIWI_0/630
1700    ./uploads.d/wsprnet.d/spots.d/K8NVH=X_EN82lm/KIWI_0/80
9520    ./uploads.d/wsprnet.d/spots.d/K8NVH=X_EN82lm/KIWI_0/10
34356   ./uploads.d/wsprnet.d/spots.d/K8NVH=X_EN82lm/KIWI_0/40
1764    ./uploads.d/wsprnet.d/spots.d/K8NVH=X_EN82lm/KIWI_0/30
996     ./uploads.d/wsprnet.d/spots.d/K8NVH=X_EN82lm/KIWI_0/2200
1716    ./uploads.d/wsprnet.d/spots.d/K8NVH=X_EN82lm/KIWI_0/160
51744   ./uploads.d/wsprnet.d/spots.d/K8NVH=X_EN82lm/KIWI_0
51748   ./uploads.d/wsprnet.d/spots.d/K8NVH=X_EN82lm
51752   ./uploads.d/wsprnet.d/spots.d
51760   ./uploads.d/wsprnet.d
68      ./uploads.d/wsprdaemon.d/noise.d/K8NVH=X_EN82lm/KIWI_0/630
68      ./uploads.d/wsprdaemon.d/noise.d/K8NVH=X_EN82lm/KIWI_0/80
68      ./uploads.d/wsprdaemon.d/noise.d/K8NVH=X_EN82lm/KIWI_0/10
68      ./uploads.d/wsprdaemon.d/noise.d/K8NVH=X_EN82lm/KIWI_0/40
68      ./uploads.d/wsprdaemon.d/noise.d/K8NVH=X_EN82lm/KIWI_0/30
68      ./uploads.d/wsprdaemon.d/noise.d/K8NVH=X_EN82lm/KIWI_0/2200
68      ./uploads.d/wsprdaemon.d/noise.d/K8NVH=X_EN82lm/KIWI_0/160
480     ./uploads.d/wsprdaemon.d/noise.d/K8NVH=X_EN82lm/KIWI_0
484     ./uploads.d/wsprdaemon.d/noise.d/K8NVH=X_EN82lm
488     ./uploads.d/wsprdaemon.d/noise.d
2772    ./uploads.d/wsprdaemon.d/spots.d/K8NVH=X_EN82lm/KIWI_0/630
27792   ./uploads.d/wsprdaemon.d/spots.d/K8NVH=X_EN82lm/KIWI_0/80
19516   ./uploads.d/wsprdaemon.d/spots.d/K8NVH=X_EN82lm/KIWI_0/10
91996   ./uploads.d/wsprdaemon.d/spots.d/K8NVH=X_EN82lm/KIWI_0/40
25852   ./uploads.d/wsprdaemon.d/spots.d/K8NVH=X_EN82lm/KIWI_0/30
2020    ./uploads.d/wsprdaemon.d/spots.d/K8NVH=X_EN82lm/KIWI_0/2200
14740   ./uploads.d/wsprdaemon.d/spots.d/K8NVH=X_EN82lm/KIWI_0/160
184692  ./uploads.d/wsprdaemon.d/spots.d/K8NVH=X_EN82lm/KIWI_0
184696  ./uploads.d/wsprdaemon.d/spots.d/K8NVH=X_EN82lm
184700  ./uploads.d/wsprdaemon.d/spots.d
185196  ./uploads.d/wsprdaemon.d
237536  ./uploads.d
1024    ./bin
8       ./hashtable.d/K8NVH_HF_0/40
12      ./hashtable.d/K8NVH_HF_0
8       ./hashtable.d/K8NVH_HF_1/30
12      ./hashtable.d/K8NVH_HF_1
8       ./hashtable.d/K8NVH_MF_0/160
12      ./hashtable.d/K8NVH_MF_0
4       ./hashtable.d/K8NVH_LF_0/2200
8       ./hashtable.d/K8NVH_LF_0
8       ./hashtable.d/K8NVH_HF_3/80
12      ./hashtable.d/K8NVH_HF_3
4       ./hashtable.d/K8NVH_MF_1/630
8       ./hashtable.d/K8NVH_MF_1
8       ./hashtable.d/K8NVH_HF_2/20
12      ./hashtable.d/K8NVH_HF_2
4       ./hashtable.d/KIWI_0/630
4       ./hashtable.d/KIWI_0/80
8       ./hashtable.d/KIWI_0/10
8       ./hashtable.d/KIWI_0/20
8       ./hashtable.d/KIWI_0/40
8       ./hashtable.d/KIWI_0/30
4       ./hashtable.d/KIWI_0/2200
4       ./hashtable.d/KIWI_0/160
52      ./hashtable.d/KIWI_0
132     ./hashtable.d
4       ./wsprdaemon/.git/branches
4       ./wsprdaemon/.git/refs/tags
8       ./wsprdaemon/.git/refs/heads
8       ./wsprdaemon/.git/refs/remotes/origin
12      ./wsprdaemon/.git/refs/remotes
28      ./wsprdaemon/.git/refs
420     ./wsprdaemon/.git/objects/pack
4       ./wsprdaemon/.git/objects/info
428     ./wsprdaemon/.git/objects
8       ./wsprdaemon/.git/logs/refs/heads
8       ./wsprdaemon/.git/logs/refs/remotes/origin
12      ./wsprdaemon/.git/logs/refs/remotes
24      ./wsprdaemon/.git/logs/refs
32      ./wsprdaemon/.git/logs
52      ./wsprdaemon/.git/hooks
8       ./wsprdaemon/.git/info
576     ./wsprdaemon/.git
976     ./wsprdaemon
4       ./kiwiclient/.git/branches
4       ./kiwiclient/.git/refs/tags
8       ./kiwiclient/.git/refs/heads
8       ./kiwiclient/.git/refs/remotes/origin
12      ./kiwiclient/.git/refs/remotes
28      ./kiwiclient/.git/refs
552     ./kiwiclient/.git/objects/pack
4       ./kiwiclient/.git/objects/info
560     ./kiwiclient/.git/objects
8       ./kiwiclient/.git/logs/refs/heads
8       ./kiwiclient/.git/logs/refs/remotes/origin
12      ./kiwiclient/.git/logs/refs/remotes
24      ./kiwiclient/.git/logs/refs
32      ./kiwiclient/.git/logs
48      ./kiwiclient/.git/hooks
8       ./kiwiclient/.git/info
704     ./kiwiclient/.git
8       ./kiwiclient/tools
100     ./kiwiclient/kiwi/__pycache__
216     ./kiwiclient/kiwi
212     ./kiwiclient/mod_pywebsocket/__pycache__
468     ./kiwiclient/mod_pywebsocket
8       ./kiwiclient/test
1576    ./kiwiclient
8       ./signal_levels/RTL/10
16      ./signal_levels/RTL/20
28      ./signal_levels/RTL
28924   ./signal_levels/KIWI_0/630
28912   ./signal_levels/KIWI_0/80
30788   ./signal_levels/KIWI_0/10
308     ./signal_levels/KIWI_0/20
30852   ./signal_levels/KIWI_0/40
28912   ./signal_levels/KIWI_0/30
27600   ./signal_levels/KIWI_0/2200
28892   ./signal_levels/KIWI_0/160
205192  ./signal_levels/KIWI_0
205224  ./signal_levels
8       ./noise_plot
470312  .

Here are my SIGNAL_LEVEL variables in wsprdaemon.conf:
SIGNAL_LEVEL_UPLOAD="noise"         SIGNAL_LEVEL_UPLOAD_MODE="proxy" 
SIGNAL_LEVEL_UPLOAD_ID="K8NVH/X" 
SIGNAL_LEVEL_UPLOAD_GRAPHS="yes"

Any suggestions?

Thanks!


Christopher D. Morgan, P.E., Ph.D., INCE Bd. Cert.
586-246-0692 (mobile)
http://www.linkedin.com/pub/christopher-morgan/7/103/b45



--
Rob Robinett
AI6VN
mobile: +1 650 218 8896


Rob Robinett
 
Edited

Yes, those are the cached spots from when your Pi couldn't reach wsprnet.org
One WD has reconnected WD will upload in batches of up to 1000 the oldest cached spots until it catches up with real time.
In WD 3.0 I have fixed and enhanced that cache upload service and also made it easier to view the operation of that upload service.

On Tue, Nov 9, 2021 at 12:41 PM Chris M <nvhengr@...> wrote:
Hi Rob,
 
Once again, I notice spots are not showing up.  I think it is because a network cable came loose some weeks ago :-/  I think it is fixed and I think in a few hours I will start seeing spots and I think the system right now is uploading a rather large backlog...
 
There are a lot of files in ~/wsprdaemon/uploads.d/ which look like "K8NVH_211019_2159_03.tbz" .  Are these temporary files which will be removed at some point?  Or is it something I should monitor so they do not use up all of the disk space?
 
Thanks!
-chris, K8NVH
 
 

 

 
--
Rob Robinett
AI6VN
mobile: +1 650 218 8896