DXKeeper - LotW Sync QSO and QSLs - Not Responding


Dave AA6YQ
 

Thanks Dave, will keep plugging.

+ Have you tried invoking an LoTW Sync operation and letting it run to completion (ignoring the "not responding" message in DXKeeper's Main window's title bar)?

73,

Dave, AA6YQ


Jon Gefaell
 

I always appreciate your input, Joe. Your contributions are vital in supporting DX Lab Suite users. Thank you very much for caring to respond to my missives.

I've monitored the process table and the resources being allocated and utilized. There's no mystery there, and every drop is accounted for, I assure you. 

The thing is, I've designed, built, and operated high-volume, high visibility technical operations infrastructures for decades. It was my career. Despite not being proficient with Windows to a minor fraction of a degree as I am with *nix variants, I'm quite well versed in networking. I run a Ubiquity UniFi Software Defined Network, and I do deep packet inspection with IPS/IDS and remote logging to a pair of Linux hosts (*). I analyze those logs periodically. I'm very aware of what source and targets are trafficking, on what ports, and at what times. Anything suspicious is detected and mitigated when necessary. 

My ham shack systems resources are well managed and allocated. I built it to be used and with ample headroom, as I've noted. My system runs very smoothly with no apparent issues. I have no malware infestation, no botnet participation, no untracked I/O or processes. I run a pretty tight ship for decades. It's what I do.

The issue isn't related to resource consumption or malware. It does seem like a resource blocking issue, however. As I said, it's my cross to bear, and one day perhaps I will resolve it. It's a corner case for sure. 

Very best regards! Stay warm, be well, and good DX!

(*) These used to be hosted on my own physical machines located in datacenters with tier 1 networks that routed my legacy class C direct assigned network 198.147.203.0/24.  It helps to have friends in the business. Now, they're virtual machines hosted by two separate providers. They run DNS, Email, IP tunnels, and other good things for myself and my friends. 


On Sun, Feb 28, 2021 at 06:47 AM, Joe Subich, W4TV wrote:

Looking at the high CPU consumers in Task Manager should help to
identify the suspect processes.


Joe Subich, W4TV
 

On 2021-02-27 10:36 PM, Jon Gefaell wrote:

However, my CPU consumption isn't implicated as the cause of the reported issue, however. And my system runs very pleasingly fast in
all aspects. Nothing 'crawls'. It does all these things at once with
ease. I built it to use it. :)
No, but it indicates a process running in the background that can be
the cause of your issue. I would speculate that something is actively
using a [high volume] internet connection - possibly malware that is
involved in some distributed processing application e.g., "Bitcoin
mining", SETI@Home or a botnet involved in DDOS activities.

Looking at the high CPU consumers in Task Manager should help to
identify the suspect processes.

73,

... Joe, W4TV


On 2021-02-27 10:36 PM, Jon Gefaell wrote:
On Sat, Feb 27, 2021 at 07:10 PM, Dave AA6YQ wrote:
I, w6de, run an i7 6-core system, NVME SSD, 16 GB memory and I
almost never see 40% CPU utilization. I see peaks of CPU use only
when running WSJTX at its end of receive decoding. My quote of Jon
in item two clearly points to a software conflict or undetected
malware infestation. Actively consuming CPU at the time were HDSDR (1.66Mhz spread, tuned to 3885), JTalert & WSJT-X (actively decoding on 7,074kHz), Chrome (viewing local 2K video stream rendered with Plex media server), a background file-sharing application, and perhaps others. 40% was the peak utilization noted observing for a minute or two.
However, my CPU consumption isn't implicated as the cause of the reported issue, however. And my system runs very pleasingly fast in all aspects. Nothing 'crawls'.  It does all these things at once with ease. I built it to use it. :)


Hank
 

Thanks Dave, will keep plugging.

-----Original Message-----
From: DXLab@groups.io <DXLab@groups.io> On Behalf Of Dave AA6YQ
Sent: Saturday, February 27, 2021 8:59 PM
To: DXLab@groups.io
Subject: Re: [DXLab] DXKeeper - LotW Sync QSO and QSLs - Not Responding

@ AA6YQ comments below

+ If you (understandably) cannot diagnose Windows yourself, then I suggest engaging with Microsoft Support to obtain their assistance in tracking down the root cause of this behavior.

I can't help but chuckle at the notion. "Hello, DX Keeper is has failures to sync LoTW QSLs for me but works for others." ;)

@ Try "I have an application that downloads information from a URL. When I run this application immediately after booting Windows, it succeeds. When I run this application after starting other applications, Windows announces that the application has stopped working".

@ You can provide them with the URL if you temporarily enable the "PC has no internet connection" option I mentioned earlier.

@ You can also inform them that the application uses Microsoft's urlmon.dll component to download the information.

73,

Dave, AA6YQ


Jon Gefaell
 

On Sat, Feb 27, 2021 at 07:10 PM, Dave AA6YQ wrote:
I, w6de, run an i7 6-core system, NVME SSD, 16 GB memory and I almost never see 40% CPU utilization. I see peaks of CPU use only when running WSJTX at its end of receive decoding. My quote of Jon in item two clearly points to a software conflict or undetected malware infestation.

Actively consuming CPU at the time were HDSDR (1.66Mhz spread, tuned to 3885), JTalert & WSJT-X (actively decoding on 7,074kHz), Chrome (viewing local 2K video stream rendered with Plex media server), a background file-sharing application, and perhaps others. 40% was the peak utilization noted observing for a minute or two.

However, my CPU consumption isn't implicated as the cause of the reported issue, however. And my system runs very pleasingly fast in all aspects. Nothing 'crawls'.  It does all these things at once with ease. I built it to use it. :) 


Dave AA6YQ
 

+ AA6YQ comments below

There is something clearly wrong in Jon’s software configuration given the following information Jon Getfall has provided:

Item one: “I run an eight-core i7 with 64GB of RAM and have over 1GB of free space on my NVME SSD's. Resource monitoring shows vast amounts of headroom in terms of processor, network bandwidth, and memory utilization. I have no issues with my system and the many things it does. So, it's elusive. I'm going with a latent defect in Windows, but what is triggering it is... inexplicable.”


Item two: “Memory is 41% utilized, CPU is 40%, Disk and Network are 0%, and GPU is 8%. I don't have the skills to look into the number of open file handles, etc. But there are no other evident issues with closing, opening, and using numerous applications that would consume additional resources, open file handles and network connections, allocating memory, etc.”

Item three: “I never, ever run anti-malware software. They're the main source of problems. IMO, but YMMV, of course. I do run regular standalone scans and always come up clean. The built-in Windows 10 firewall does its thing, and my Unifi Software Defined Network's firewall is doing a lovely job as well. But agreed, what's in place is in place when it works and when it doesn't.”


I, w6de, run an i7 6-core system, NVME SSD, 16 GB memory and I almost never see 40% CPU utilization. I see peaks of CPU use only when running WSJTX at its end of receive decoding. My quote of Jon in item two clearly points to a software conflict or undetected malware infestation.

+ Excellent point, Dave! The 40% CPU consumption is a definitely red flag. The Windows Task Manager should reveal the source of this.

73,

Dave, AA6YQ


w6de
 

There is something clearly wrong in Jon’s software configuration given the following information Jon Getfall has provided:

Item one: “I run an eight-core i7 with 64GB of RAM and have over 1GB of free space on my NVME SSD's. Resource monitoring shows vast amounts of headroom in terms of processor, network bandwidth, and memory utilization. I have no issues with my system and the many things it does. So, it's elusive. I'm going with a latent defect in Windows, but what is triggering it is... inexplicable.”

 

Item two: “Memory is 41% utilized, CPU is 40%, Disk and Network are 0%, and GPU is 8%. I don't have the skills to look into the number of open file handles, etc. But there are no other evident issues with closing, opening, and using numerous applications that would consume additional resources, open file handles and network connections, allocating memory, etc.”

 

Item three: “I never, ever run anti-malware software. They're the main source of problems. IMO, but YMMV, of course. I do run regular standalone scans and always come up clean. The built-in Windows 10 firewall does its thing, and my Unifi Software Defined Network's firewall is doing a lovely job as well. But agreed, what's in place is in place when it works and when it doesn't.”

 

I, w6de, run an i7 6-core system, NVME SSD, 16 GB memory and I almost never see 40% CPU utilization.  I see peaks of CPU use only when running WSJTX at its end of receive decoding.  My quote of Jon in item two clearly points to a software conflict or undetected malware infestation. 

 

Have you read this DXLab WIKI article and followed its advice?

http://www.dxlabsuite.com/dxlabwiki/TerminateUnnecessaryApplications?highlight=%28programs%29

 

73,

Dave, w6de

 

From: DXLab@groups.io [mailto:DXLab@groups.io] On Behalf Of Jon Gefaell
Sent: Sunday, February 28, 2021 00:01
To: DXLab@groups.io
Subject: Re: [DXLab] DXKeeper - LotW Sync QSO and QSLs - Not Responding

 

On Sat, Feb 27, 2021 at 02:05 PM, Dave AA6YQ wrote:

+ For the record, I frequently see the "not responding" message in window title bars -- including in the title bars of Microsoft applications. Letting the operation run to completion invariably succeeds.

Thank you, Dave. I just did this again so that I could explain it carefully. Hopefully, this is useful at some point. I'm delighted to be discussing this, thank you.

I selected "Sync LOTW QSLs." The progress dialogue pops up and halts at 59 seconds. The title bar displays 'not responding,' and shortly thereafter, the windows dialogue comes up, saying the same thing and offering a choice of stopping the program or waiting. I select to wait. The DXKeeper windows (main, capture, and the QSL sync progress) are shaded. I have been waiting 35 minutes with no change.

Memory is 41% utilized, CPU is 40%, Disk and Network are 0%, and GPU is 8%. I don't have the skills to look into the number of open file handles, etc. But there are no other evident issues with closing, opening, and using numerous applications that would consume additional resources, open file handles and network connections, allocating memory, etc. 

I'll keep at it. It feels like a blocking issue. One day I'll find my gremlin. :) Thanks again, sir!

 


Dave AA6YQ
 

@ AA6YQ comments below

+ If you (understandably) cannot diagnose Windows yourself, then I suggest engaging with Microsoft Support to obtain their assistance in tracking down the root cause of this behavior.

I can't help but chuckle at the notion. "Hello, DX Keeper is has failures to sync LoTW QSLs for me but works for others." ;)

@ Try "I have an application that downloads information from a URL. When I run this application immediately after booting Windows, it succeeds. When I run this application after starting other applications, Windows announces that the application has stopped working".

@ You can provide them with the URL if you temporarily enable the "PC has no internet connection" option I mentioned earlier.

@ You can also inform them that the application uses Microsoft's urlmon.dll component to download the information.

73,

Dave, AA6YQ


Jon Gefaell
 

On Sat, Feb 27, 2021 at 04:47 PM, Dave AA6YQ wrote:

+ The fact that Windows displays a separate window offering to stop DXKeeper or continue waiting is also unique to your situation.

I look forward to hearing back from the OP whether waiting has resolved his concern. Or maybe I missed that?

+ You have reported that the LoTW Sync functions work if you start DXKeeper immediately after rebooting Windows; that likely rules out interference from anti-malware or firewall applications automatically started by Windows, since they were running when you reported success. 

Agreed. I never, ever run anti-malware software. They're the main source of problems. IMO, but YMMV, of course. I do run regular standalone scans and always come up clean. The built-in Windows 10 firewall does its thing, and my Unifi Software Defined Network's firewall is doing a lovely job as well. But agreed, what's in place is in place when it works and when it doesn't. 

+ If you (understandably) cannot diagnose Windows yourself, then I suggest engaging with Microsoft Support to obtain their assistance in tracking down the root cause of this behavior.

I can't help but chuckle at the notion. "Hello, DX Keeper is has failures to sync LoTW QSLs for me but works for others." ;)

The *only* reason I run Windows, which I despise, is because of amateur radio. With DX Lab Suite square at the center. I couldn't even consider anything else.

I think this is my cross to bear, on my own. As problems go, it's pretty mild. Grin. 

Thanks as always for your efforts.


Dave AA6YQ
 

+ AA6YQ comments below

Thank you, Dave. I just did this again so that I could explain it carefully. Hopefully, this is useful at some point. I'm delighted to be discussing this, thank you.

I selected "Sync LOTW QSLs." The progress dialogue pops up and halts at 59 seconds. The title bar displays 'not responding,' and shortly thereafter, the windows dialogue comes up, saying the same thing and offering a choice of stopping the program or waiting. I select to wait. The DXKeeper windows (main, capture, and the QSL sync progress) are shaded. I have been waiting 35 minutes with no change.

Memory is 41% utilized, CPU is 40%, Disk and Network are 0%, and GPU is 8%. I don't have the skills to look into the number of open file handles, etc. But there are no other evident issues with closing, opening, and using numerous applications that would consume additional resources, open file handles and network connections, allocating memory, etc. 

I'll keep at it. It feels like a blocking issue. One day I'll find my gremlin. :) Thanks again, sir!

+ The fact that Windows displays a separate window offering to stop DXKeeper or continue waiting is also unique to your situation..

+ You have reported that the LoTW Sync functions work if you start DXKeeper immediately after rebooting Windows; that likely rules out interference from anti-malware or firewall applications automatically started by Windows, since they were running when you reported success. 

+ If you (understandably) cannot diagnose Windows yourself, then I suggest engaging with Microsoft Support to obtain their assistance in tracking down the root cause of this behavior.

        73,

                Dave, AA66YQ

 


Jon Gefaell
 

On Sat, Feb 27, 2021 at 02:05 PM, Dave AA6YQ wrote:
+ For the record, I frequently see the "not responding" message in window title bars -- including in the title bars of Microsoft applications. Letting the operation run to completion invariably succeeds.

Thank you, Dave. I just did this again so that I could explain it carefully. Hopefully, this is useful at some point. I'm delighted to be discussing this, thank you.

I selected "Sync LOTW QSLs." The progress dialogue pops up and halts at 59 seconds. The title bar displays 'not responding,' and shortly thereafter, the windows dialogue comes up, saying the same thing and offering a choice of stopping the program or waiting. I select to wait. The DXKeeper windows (main, capture, and the QSL sync progress) are shaded. I have been waiting 35 minutes with no change.

Memory is 41% utilized, CPU is 40%, Disk and Network are 0%, and GPU is 8%. I don't have the skills to look into the number of open file handles, etc. But there are no other evident issues with closing, opening, and using numerous applications that would consume additional resources, open file handles and network connections, allocating memory, etc. 

I'll keep at it. It feels like a blocking issue. One day I'll find my gremlin. :) Thanks again, sir!

 


Dave AA6YQ
 

+ AA6YQ comments below

* That's a likely symptom of resource exhaustion, Windows misconfiguration, or a latent defect in Windows - which is why I posted the above advice.

I've re-read both my description and his. They seem complimentary. There may be an issue of semantics, and my choice of words to describe the issue may benefit from fine-tuning. I would appreciate some insight as to how our two reports are crucially different.

+ Hank reported the appearance of "not responding" in DXKeeper's Main window's title bar, and (understandably) misinterpreted it to mean that the Sync process was not proceeding. My advice is to let the operation run to completion, ignoring the "not responding" message.

+ You reported that unless you start DXKeeper alone after rebooting, DXKeeper never completes the Sync process. The fact that the Sync process runs to completion when your instance of Windows has been freshly rebooted is what points to a resource management or misconfiguration issue in Windows.

+ For the record, I frequently see the "not responding" message in window title bars -- including in the title bars of Microsoft applications. Letting the operation run to completion invariably succeeds.

73,

Dave. AA6YQ


Jon Gefaell
 

Thanks, Dave. I greatly appreciate your advice. I prefer my workaround of rebooting and starting DXKeeper by itself, performing the sync, and starting the rest of the software I use. I really like the integration and automation that is provided by DXKeeper. My system reboots in about 20-30 seconds, so it's no problem. :)

On Sat, Feb 27, 2021 at 01:46 PM, Dave AA6YQ wrote:

# Until you diagnose and repair your system, here's a workaround to consider when you want to "Sync LoTW QSLs":


Jon Gefaell
 

On Sat, Feb 27, 2021 at 01:39 PM, Dave AA6YQ wrote:
* That's a likely symptom of resource exhaustion, Windows misconfiguration, or a latent defect in Windows - which is why I posted the above advice.

I've re-read both my description and his. They seem complimentary. There may be an issue of semantics, and my choice of words to describe the issue may benefit from fine-tuning. I would appreciate some insight as to how our two reports are crucially different.

 

At any rate, as I stated, however:

"Something your and my systems are doing is causing the issue, but it's elusive and does not seem to be experienced by others. This certainly seems to explicate DX Keeper."

I run an eight-core i7 with 64GB of RAM and have over 1GB of free space on my NVME SSD's. Resource monitoring shows vast amounts of headroom in terms of processor, network bandwidth, and memory utilization. I have no issues with my system and the many things it does. So, it's elusive. I'm going with a latent defect in Windows, but what is triggering it is... inexplicable.

It's a small bother to work around the issue, and certainly not a DX Keeper defect. I wish I was nearly as handy with system analysis in Windows as I am with *nix. I'd be all over it.  Thanks for your wonderful software and the effort you put into ensuring it interoperates with everything so well. There is nothing that comes close.


Dave AA6YQ
 

# Even more AA6YQ comments below

You're not alone. I reported this same issue in this thread https://groups.io/g/DXLab/topic/78720234#197922

and was advised:

+ Your next step should be to diagnose and correct your Windows system.

* Your report in the above-cited thread is entirely different. In that thread, you reported

"This only seems to happen when I have the entire DX Lab Suite of programs up and running. If I start from a fresh reboot, and only start DX Keeper I can successfully perform the sync functions. Sometimes merely closing the other DX Lab Suite programs will result in the sync functions working."

* That's a likely symptom of resource exhaustion, Windows misconfiguration, or a latent defect in Windows - which is why I posted the above advice.


# Until you diagnose and repair your system, here's a workaround to consider when you want to "Sync LoTW QSLs":

1. check the "PC has no internet connection" box on the "QSL Configuration" window's LotW tab

2. follow the instructions in the "Obtaining New Confirmations from LoTW" section of " Uploading to LotW and Synchronizing with LotW from a PC not connected to the internet" in

https://www.dxlabsuite.com/dxlabwiki/LotWNoInternetAccess

3. use your web browser to download the file from LoTW and save it to the specified location, and then click the OK button to trigger DXKeeper's processing of that information.

4. uncheck the "PC has no internet connection" box on the "QSL Configuration" window's LotW tab

73,

Dave, AA6YQ


Dave AA6YQ
 

* AA6YQ comments below

You're not alone. I reported this same issue in this thread https://groups.io/g/DXLab/topic/78720234#197922

and was advised:

+ Your next step should be to diagnose and correct your Windows system.

* Your report in the above-cited thread is entirely different. In that thread, you reported

"This only seems to happen when I have the entire DX Lab Suite of programs up and running. If I start from a fresh reboot, and only start DX Keeper I can successfully perform the sync functions. Sometimes merely closing the other DX Lab Suite programs will result in the sync functions working."

* That's a likely symptom of resource exhaustion, Windows misconfiguration, or a latent defect in Windows - which is why I posted the above advice.

73,

Dave, AA6YQ


Jon Gefaell
 

You're not alone. I reported this same issue in this thread https://groups.io/g/DXLab/topic/78720234#197922

and was advised:

+ Your next step should be to diagnose and correct your Windows system.

There do not appear to be any apparent issues with my Windows system, however. Other than Windows being Windows. LOL

Instead of syncing whenever I like, I reboot, start only DX Keeper, do the sync which usually works, then start up the rest of my software. IIWII.

However, it is nice to see someone else is having this issue, just from the perspective of independent replication. The trick would be to troubleshoot the behavior somehow. I could trace system calls in Unix, but I am not proficient at this on Windows. 

Something your and my systems are doing is causing the issue, but it's elusive and does not seem to be experienced by others. This certainly seems to explicate DX Keeper.  


On Sat, Feb 27 2021, at 11:05 AM, Hank wrote:

Problem:  When running LotW QSO or QSL sync, get:  "DXKeeper online QSO or QSL) sync (not responding)." This requires closing DXKeeper to kill the sync window.
 
Closing and reopening DXKeeper continues to fault.  If I reboot the computer the syncs function for a few times then eventually get the same, "not responding."  Another strange one, when the syncs function following a computer reboot, each sync of QSO and QSL LotW always responds with 1 QSO or QSL synced.


Dave AA6YQ
 

+ AA6YQ commengts below
Hey folks.  Been using DXLab for years, and finally hit an issues I could resolve.  Looking for Devine guidance:
 
Problem:  When running LotW QSO or QSL sync, get:  "DXKeeper online QSO or QSL) sync (not responding)." This requires closing DXKeeper to kill the sync window.
 

+ The words "not responding" appearing in a window's title bar is Microsoft-speak for "waiting for input or output"; in this case, DXKeeper is waiting for LoTW to download the requested data. Let it finish!

+ DXKeeper uses a Microsoft component (URLMON.dll) to do the downloading. Unfortunately, this component does not like being aborted while an operation is underway.

         Dave, AA6YQ


Hank
 

Hey folks.  Been using DXLab for years, and finally hit an issues I could resolve.  Looking for Devine guidance:
 
Problem:  When running LotW QSO or QSL sync, get:  "DXKeeper online QSO or QSL) sync (not responding)." This requires closing DXKeeper to kill the sync window.
 
Closing and reopening DXKeeper continues to fault.  If I reboot the computer the syncs function for a few times then eventually get the same, "not responding."  Another strange one, when the syncs function following a computer reboot, each sync of QSO and QSL LotW always responds with 1 QSO or QSL synced.
 
Confirmed all LotW settings are good.  Able to login to LotW - no issue.
 
DXKeeper and TQSL are allowed in/out through windows 10  firewall.
 
Restored DXKeeper - no change
 
EQSL syncs with no issues.
 
Appreciate any suggestions