Topics

feature request: spots lifetime in minutes instead of hours

Frank Grossmann
 

Good morning all, hi Dave,

 

I’d love to see the spots lifetime settable in commander bandspread to be settable in minutes instead of hours.

 

I pretty much operate CW only and use RBN spots. As most OPs don’t stay on frequency for a whole hour most of the displayed spots I see in bandspread / Flex SmartSDR are not accurate.

Spots get refreshed by the RBN network every couple of minutes so for me a spots lifetime of 10 minutes would be great.

 

Thanks

Frank

 

 

 

Von: DXLab@groups.io <DXLab@groups.io> Im Auftrag von Danny Douglas
Gesendet: Samstag, 10. November 2018 16:31
An: DXLab@groups.io
Betreff: Re: [DXLab] Compacting spot databases: new information

 

Thought I would take a look at spots, when I turn on WSJT, which automatically sets those spots over to SpotCollector.   Before turning on WSJT, I let SC run with no filtering on 17 meters, and received 65 spots, in 30 minutes.    I then turned on my WSJT so that its spots would also be sent across to SC, and checked again 30 minutes later.  That resulted in 175 spots (a great percentage of my own receiver spotting the WSJT stations, over and over again.  This could really result in a lot of spots in the

 

It did not happen this time, but on occasion I do see a spot on SC, for something I need on a bands digital modes.  That is really what I was looking for here , to see if they showed up in Red as a needed band/mode. So am assuming, if I am able to filter two things at once (band, and need) it would cut down on those shown spots.  But, I guess the question is: are those unseeded WSJT spots, that are being sent to SC, actually being collected, and taking up data space someplace, when I am filtering for NEED only.  I.E. Can they jam up the system, when not shown actually needed?

 

 

N7DC@...
Ex WN5QMX,WA5UKR,ET2US,ET3USA,SV0WPP,VS6DD,N7DC/YV5/G5CTB
QSL Bureau, DIRECT, LOTW Preferred, eQSL used but upload at a courtesy only, as do not use the system for awards.

On November 10, 2018 at 9:26 AM Dave AA6YQ <aa6yq@...> wrote:

+ AA6YQ comments below

On Sat, Nov 10, 2018 at 04:42 AM, John Pelham wrote:

In the past few days I've experimented with SC and its spot database (db). (To make bad things happen in SC more quickly, I added not only W9PA-4, but W9PA-5 to my spot sources. W9PA-5 is a real firehose of FT8 spots.) I've come to these conclusions.

 

As spots accumulate in the db, db size increases. Also, the space used in the db per entry increases as the overall db size increases. (This is counter to my non-expert expectation. I would think that db structures and indices, etc., would increase in size more slowly than the data itself as entries accumulate, so the db space per entry should decrease as the db size increases.) As the db size and size/entry increases, the db performance decreases. Queue flushing (the countdown of the Q: number in the SC window) proceeds more slowly. Eventually (when the db size reaches several hundred MB), queue flushing slows down enough so that it can't keep up; the queue overflows, and keeps overflowing because the speed at which the entries can be written to the db has become drastically reduced (to tens or hundreds of milliseconds per entry, it looks like, from watching the Q: countdown). At this point SC also displays the other issues I mentioned previously: unresponsiveness to user interactions with the UI, increased CPU usage, CQDX connection problems.

 

An additional observation: I tested with pruning turned off (unchecked Prune db on startup, unchecked Prune db hourly, Prune entries older than set to 7), and I saw no difference in the behaviors I described above. Pruning, or not, makes no difference. Whether db space is consumed by actual entries or deleted entries appears to make no difference. Without compaction, pruning helps not at all with db size or performance.

 

In my OP on the original thread, I said "Shouldn't there be an option to automatically compact these databases? Maybe every 5 minutes, or every hour, or user choice of timing?" I have a theory that frequent compaction, combined with a reasonable Prune entries older than setting, would prevent the db from ever getting too big and too slow. I'd like to test this theory, but manually compacting every few minutes for hours would be difficult. Is there any possibility that functionality to automatically compact the db periodically could be added, if only as a test, for now? On my system compacting the db takes one second or less and appears to cause no performance impact to any other processes. I'd rather compact too often than too seldom. Five minutes still seems like a good interval.

+ The Spot Database grows because SpotCollector creates new Spot Database Entries when spot arrive bearing a callsign for which no Entry already exists in the spotted mode near the spotted frequency.

+ With automatic Pruning disabled, the Spot Database would grow without bound.

+ With automatic Pruning enabled, SpotCollector conducts an hourly review of every Spot Database Entry, deleting those whose age exceeds the value you've specified. While the deleting an Entry makes it invisible, it does not free the space consumed by the Entry, or the space consumed by the data structures that the database engine employs to provide rapid access to the Entries.

+ The database engine provides a Compact operation that effectively rebuilds the database without the delete Entries. In the current version of SpotCollector, the Compact operation can be invoked manually be clicking the Cmpct button; it is invoked automatically by the Pruning function if the Spot Database size exceeds 1 gigabyte. With today's high-volume spot sources, this is not effective.

+ I have been assessing schemes for automating more frequent compaction. Keeping track of the number of Pruned Spot Database Entries is effective: after each hourly Pruning, if the cumulative number of deleted Spot Database Entries exceeds 1000, Compaction is autopmatically invoked.

+ If spot sources continue to increase in spot volume, it may be necessarily to enable Pruning at shorter intervals than hourly.

        73,
           
             Dave, AA6YQ

 

Dave AA6YQ
 

+ AA6YQ comments below

I’d love to see the spots lifetime settable in commander bandspread to be settable in minutes instead of hours.

+ As stated in the 4th bullet in

<https://www.dxlabsuite.com/commander/Help/Configuration.htm#DX_Spot_settings>

+ you can set the Lifetime to a fraction of an hour, for example .17 for 10 minutes.

73,

Dave, AA6YQ

Frank Grossmann
 

Great! Thanks - 73


-----Ursprüngliche Nachricht-----
Von: DXLab@groups.io <DXLab@groups.io> Im Auftrag von Dave AA6YQ
Gesendet: Montag, 24. Juni 2019 07:15
An: DXLab@groups.io
Betreff: Re: [DXLab] feature request: spots lifetime in minutes instead of hours

+ AA6YQ comments below

I’d love to see the spots lifetime settable in commander bandspread to be settable in minutes instead of hours.

+ As stated in the 4th bullet in

<https://eur02.safelinks.protection.outlook.com/?url=https%3A%2F%2Fwww.dxlabsuite.com%2Fcommander%2FHelp%2FConfiguration.htm%23DX_Spot_settings&amp;data=02%7C01%7C%7C35337c60ab24485cef4208d6f862f588%7C84df9e7fe9f640afb435aaaaaaaaaaaa%7C1%7C0%7C636969501236366066&amp;sdata=7I%2Fku5%2FnqFw7dEYIANwqz2Rf1tlX6b4B5iOPjrS%2B3vA%3D&amp;reserved=0>

+ you can set the Lifetime to a fraction of an hour, for example .17 for 10 minutes.

73,

Dave, AA6YQ