Spot Collector Sluggish and slowing everything down
This article in the DXLab WIKI outlines several things you can do to improve SpotCollector performance.
http://www.dxlabsuite.com/dxlabwiki/OptimizeSCPerformance
You might start at limiting the size of the database file by Pruning the database more frequently.
73,
Dave, w6de
Sent: Sunday, March 28, 2021 18:11
To: DXLab Reflector
Subject: [DXLab] Spot Collector Sluggish and slowing everything down
Hello
Once i start Spot Collector, seems like everything slows down considerable and response time on other modules i.e. logging, etc. become noticeably sluggish, unresponsive in some cases
The worst response is when i try to do anything on teh spot Collector window. For example clicking on a spot sometimes take many clicks /tries for it to respond.
Checking Windows task Manager shows Spot Collector continuously hogging anywhere from 25 % to 30 % of the CPU usage.
I limited the Spot collector spot sources to only one source (N6WS), that did not make a difference
I ran the PC in safe mode an that did not make a difference either
I disabled the protection (Defender and virus scans, etc) , with no difference
I'm not sure what to try next.
I guess the question is if its normal for SC to utilize this much CPU, and if so, why does it bring things to halt even at 30% CPU usage. Even with everything running including SC, the total CPU usage is in the 50ish % range , is that too much?
73 Saad N5FF
Once i start Spot Collector, seems like everything slows down considerable and response time on other modules i.e. logging, etc. become noticeably sluggish, unresponsive in some cases
The worst response is when i try to do anything on teh spot Collector window. For example clicking on a spot sometimes take many clicks /tries for it to respond.
Checking Windows task Manager shows Spot Collector continuously hogging anywhere from 25 % to 30 % of the CPU usage.
I limited the Spot collector spot sources to only one source (N6WS), that did not make a difference
I ran the PC in safe mode an that did not make a difference either
I disabled the protection (Defender and virus scans, etc) , with no difference
I'm not sure what to try next.
I guess the question is if its normal for SC to utilize this much CPU, and if so, why does it bring things to halt even at 30% CPU usage. Even with everything running including SC, the total CPU usage is in the 50ish % range , is that too much?
+ CPU usage is only one of several factors that can slow system performance. See
https://www.dxlabsuite.com/dxlabwiki/OptimizeSCPerformance
73,
Dave, AA6YQ
Once i start Spot Collector, seems like everything slows down considerable and response time on other modules i.e. logging, etc. become noticeably sluggish, unresponsive in some cases
The worst response is when i try to do anything on teh spot Collector window. For example clicking on a spot sometimes take many clicks /tries for it to respond.
Checking Windows task Manager shows Spot Collector continuously hogging anywhere from 25 % to 30 % of the CPU usage.
I limited the Spot collector spot sources to only one source (N6WS), that did not make a difference
I ran the PC in safe mode an that did not make a difference either
I disabled the protection (Defender and virus scans, etc) , with no difference
I'm not sure what to try next.
I guess the question is if its normal for SC to utilize this much CPU, and if so, why does it bring things to halt even at 30% CPU usage. Even with everything running including SC, the total CPU usage is in the 50ish % range , is that too much?
+ CPU usage is only one of several factors that can slow system performance. See
https://www.dxlabsuite.com/dxlabwiki/OptimizeSCPerformance
73,
Dave, AA6YQ
Just to circle back on this issue with some feedback
Everything got to the point of being so sluggish, almost non responsive and unusable. During a contest with spots pouring in so fast, it was practically almost frozen.
I tried about everything recommend in the document., short of adding more memory and dedicating CPUs, the only thing that made HUGE difference is "Size control" in the SC Config / Spot database. I "cleared" it and also changed "Brune entries to older than 1 day", also checked "clear data base on startup".
This made a night and day difference, and everything was responded instantly.
+ Managing a large database requires available RAM. Instead of enabling "clear data base on startup", I suggest that you enable "Prune Spot Database hourly".
+ When seeking to understand the operating pattern of a particular station, add its callsign to SpotCollector's Special Callsign List with a NOPRUNE tag.
73,
Dave, AA6YQ
Then out of the blue, as if by inspiration, I got an email a few days ago from the operator of one of the spot servers I was connected to (NC7J). He somewhat nicely asked me if I really wanted to have a wide-open report of all spots reported from anywhere in the world, because it was causing a heavy load on the server. He sent me an example filter string I might try to reduce the number of spots I received. He said he uses DXLAB too, and he figured not only was I making a lot of unnecessary work on the spot server, but my computer was probably having a very difficult time trying to process all those spots in real time, to see which ones are new entities, new bands, new modes, or whatever.
I entered his suggested filter string on the server, so that I would only receive spots that originated in my general area of the world (western USA). I also noticed that I had left several spot servers enabled in SpotCollector, figuring the more the merrier, I suppose. They were probably flooding me with tons of useless spots too. BTW, I also reduced the size of my spot database (1 hour), and had it purge when I restart the program, and purge anything older than one hour. The final change that helped improve responsiveness was to uncheck the "Display previous QSOs on Lookup" checkbox on the DXKeeper Configuration panel. After making those changes, I noticed a HUGE HUGE difference in responsiveness. Now it's 2 or 3 or 4 seconds, instead of 10 or 20 or 30 seconds. It makes it fun to operate again. :)
I hope this might give someone else an idea or two to check, to make DXLAB work the way it's supposed to.
73,
Tom, W7GT
Hoping it might help someone else, here's my success story. I was very frustrated with how long it took DXLAB to update all the modules (DXView, Pathfinder, Spot Collector and DXKeeper) after entering a new call -- for example from clicking an entry in WSJT-X, or from clicking a call on the spot window. Sometimes it seemed like over 30 seconds, and by that time you don't care anymore. I looked into the possibility that my anti-virus program might be the culprit, but that didn't yield much change, if any, for me.
+ Did you reach this conclusion after rebooting Windows into "Safe mode with networking"? If you did not take this step, your conclusion is not reliable. See
https://www.dxlabsuite.com/dxlabwiki/ApplicationInteference
Then out of the blue, as if by inspiration, I got an email a few days ago from the operator of one of the spot servers I was connected to (NC7J). He somewhat nicely asked me if I really wanted to have a wide-open report of all spots reported from anywhere in the world, because it was causing a heavy load on the server. He sent me an example filter string I might try to reduce the number of spots I received. He said he uses DXLAB too, and he figured not only was I making a lot of unnecessary work on the spot server, but my computer was probably having a very difficult time trying to process all those spots in real time, to see which ones are new entities, new bands, new modes, or whatever.
+ To understand how SpotCollector works, review this article:
https://www.dxlabsuite.com/dxlabwiki/CollectingSpots
I entered his suggested filter string on the server, so that I would only receive spots that originated in my general area of the world (western USA).
+ If P5DX is spotted by a station in Japan, do you really want to wait until someone in your "general area of the world" spots it? Absolutel not! You want to QSY immediately and make the QSO before the cluster hordes arrive. Limit what you see to what you are likely able to work, as described here:
https://www.dxlabsuite.com/dxlabwiki/HidingDXYouCantHear
I also noticed that I had left several spot servers enabled, figuring the more the merrier, I suppose.
+ No - not "the more the merrier". While all spots eventually reach all cluster nodes, they don't do so at the same time. Experience shows that a DX station will be more quickly reported by a geographically closer cluster node than by a geographically remote cluster node -- sometimes by as much as 30-60 seconds, which is enough time to QSY and make the QSO before the competition arrives en masse. To a DXer living on the west coast of North America, I recommend configuring SpotCollector to connect to cluster nodes on North American West Coast, North American East Coast, Europe, and Asia.
There were probably flooding me with tons of useless spots too.
+ Spots of a station with whom a confirmed QSO would not advance your award progress are not necessarily useless; they reveal actual propagation. When I was pursuing ZC4GR on FT8, it was spots of stations in 4X, 5B4, SU, and TA that identified the best band openings between my QTH and ZC4. This display showed me when stations in those countries were spotted by stations within 500 miles of my QTH:
https://www.dxlabsuite.com/spotcollector/nrMe-nrZC4.jpg
+ I began monitoring the 20m FT8 sub-band between 20Z and 23Z - and eventually worked ZC4GR one day at 2034Z.
BTW, I also reduced the size of my spot database, and had it purge when I restart the program, and purge anything older than one day.
+ Not a good idea if you're trying to identify operating patterns of needed DX stations or propagation openings between needed DX stations and your QTH.
The final change that helped improve responsiveness was to uncheck the "Display previous QSOs on Lookup" checkbox on the DXKeeper Configuration panel. After making that change, I immediately noticed a HUGE HUGE difference in responsiveness. Now it's 2 or 3 or 4 seconds, instead of forever. It makes it fun to operate again. :)
+ That operation consumes zero resources unless you double-click on a Log Page Display entry or a Spot Database Display entry.
I hope this might give someone else an idea or two to check, to make DXLAB work the way it's supposed to.
+ What you're advocating is definitely not the best way to exploit SpotCollector to help you find and work the DX you need.
+ I suggest that you review these instructions for optimizing SpotCollector's performance:
https://www.dxlabsuite.com/dxlabwiki/OptimizeSCPerformance
73,
Dave, AA6YQ