Topics

Pathfinder "509 Too Many Requests" Importing Contest QSO's

 

I had my best day contesting today in the NAQP-CW and scored 161 Q's (not great but good for me).  When I tried to import them into DXKeeper, I kept getting a popup that wanted to save a file.  The files contained one line that said "509 Too Many Requests".  It appears I was overloading QRZ.com. I finally closed Pathfinder to just get the Q's loaded but now I have a bunch of log entries with no information other than from the contest.  Is this a common error?  What is the best practice for importing contest logs?  Should I set up DXKeeper to log from N1MM+ in real time?  How do I get the CBA stuff loaded into about 130 Q's?

Bob AF9W

Dave AA6YQ
 

+ AA6YQ comments below

I had my best day contesting today in the NAQP-CW and scored 161 Q's (not great but good for me). When I tried to import them into DXKeeper, I kept getting a popup that wanted to save a file.

+ What was the exact text displayed in the popup?

+ Is there an errorlog.txt file present in you DXKeeper folder?

The files contained one line that said "509 Too Many Requests".

+ To what "files" are you referring?

It appears I was overloading QRZ.com. I finally closed Pathfinder to just get the Q's loaded but now I have a bunch of log entries with no information other than from the contest. Is this a common error? What is the best practice for importing contest logs? Should I set up DXKeeper to log from N1MM+ in real time? How do I get the CBA stuff loaded into about 130 Q's?

+ By agreement with the management of QRZ, Pathfinder limits the rate at which it queries QRZ.com via Pathfinder to one callsign every 4 seconds. I am not aware of any additional limits imposed by QRZ. No one in recent memory has reported any problems with this functionality.

+ Lets determine what's going on before considering alternatives.

73,

Dave, AA6YQ

 

I'll delete the log entries and repeat the process.  If it happens again, I'll collect the data you need. 

Bob AF9W


From: DXLab@groups.io <DXLab@groups.io> on behalf of Dave AA6YQ <aa6yq@...>
Sent: Saturday, January 11, 2020, 11:25 PM
To: DXLab@groups.io
Subject: Re: [DXLab] Pathfinder "509 Too Many Requests" Importing Contest QSO's

+ AA6YQ comments below

I had my best day contesting today in the NAQP-CW and scored 161 Q's (not great but good for me).  When I tried to import them into DXKeeper, I kept getting a popup that wanted to save a file.

+ What was the exact text displayed in the popup?

+ Is there an errorlog.txt file present in you DXKeeper folder?

The files contained one line that said "509 Too Many Requests". 

+ To what "files" are you referring?

It appears I was overloading QRZ.com. I finally closed Pathfinder to just get the Q's loaded but now I have a bunch of log entries with no information other than from the contest.  Is this a common error?  What is the best practice for importing contest logs?  Should I set up DXKeeper to log from N1MM+ in real time?  How do I get the CBA stuff loaded into about 130 Q's?

+ By agreement with the management of QRZ, Pathfinder limits the rate at which it queries QRZ.com via Pathfinder to one callsign every 4 seconds. I am not aware of any additional limits imposed by QRZ. No one in recent memory has reported any problems with this functionality.

+ Lets determine what's going on before considering alternatives.

                 73,

                        Dave, AA6YQ






 

I deleted the old entries and reimported the .adi file.  Today I had 5 errors.  I sent you an email to your QRZ email address with a .zip file containing:

  • screen shots of the popup errors
  • the files that I saved when the error occurred
  • Error logs for DXKeeper and Pathfinder
  • a screen shot of one of the imported entries that received the error showing that nothing was added from QRZ.com
  • a copy of my log file before I executed the import
  • a copy of the adi file I imported. 
When the errors occur the extra fields in the log entry are not filled by from QRZ.  

Bob AF9W

 

Dave AA6YQ
 

+ AA6YQ comments below

I deleted the old entries and reimported the .adi file. Today I had 5 errors. I sent you an email to your QRZ email address with a .zip file containing:

* screen shots of the popup errors
* the files that I saved when the error occurred
* Error logs for DXKeeper and Pathfinder
* a screen shot of one of the imported entries that received the error showing that nothing was added from QRZ.com
* a copy of my log file before I executed the import
* a copy of the adi file I imported.

When the errors occur the extra fields in the log entry are not filled by from QRZ.

+ Thanks for the files, Bob.

+ The popup errors shown in your screenshots are not associated with any DXLab application. One's title bar specifies

0% of N4OGW from www.qrz.com Completed

+ and the other's title bar specifies

File Download - Security Warning

+ These look like windows displayed by an anti-malware application that is monitoring internet access. You must configure your anti-malware applications to consider Pathfinder and your other DXLab applications to be "safe" so that they do not prevent correct operation.

+ All 6 of the import error messages you forwarded are valid, caused by a "US County" that is inconsistent with the "US State". Your QSO with KE4EK, for example, specifies a state of FL and a county of KY,Fayette.

+ So the error message

"invalid secondary administrative subdivision: KY,Fayette"

+ is exactly right.

+ If you still encounter difficulties after eliminating the interference from your anti-malware application, don't hesitate to let me know.

73,

Dave, AA6YQ

 

I agree that the security message is from my anti-malware program. However, the initial pop-up is from QRZ.com. I think it is responding to the Pathfinder request to download the information for the QSO. I'll add DXLab Suite and QRZ.COM to my safe list and see if this happens again.

Bob AF9W

-----Original Message-----
From: DXLab@groups.io <DXLab@groups.io> On Behalf Of Dave AA6YQ
Sent: Sunday, January 12, 2020 12:03 PM
To: DXLab@groups.io
Subject: Re: [DXLab] Pathfinder "509 Too Many Requests" Importing Contest
QSO's

+ AA6YQ comments below

I deleted the old entries and reimported the .adi file. Today I had 5 errors. I sent
you an email to your QRZ email address with a .zip file containing:

* screen shots of the popup errors
* the files that I saved when the error occurred
* Error logs for DXKeeper and Pathfinder
* a screen shot of one of the imported entries that received the error
showing that nothing was added from QRZ.com
* a copy of my log file before I executed the import
* a copy of the adi file I imported.

When the errors occur the extra fields in the log entry are not filled by from QRZ.

+ Thanks for the files, Bob.

+ The popup errors shown in your screenshots are not associated with any
DXLab application. One's title bar specifies

0% of N4OGW from www.qrz.com Completed

+ and the other's title bar specifies

File Download - Security Warning

+ These look like windows displayed by an anti-malware application that is
monitoring internet access. You must configure your anti-malware applications
to consider Pathfinder and your other DXLab applications to be "safe" so that
they do not prevent correct operation.

+ All 6 of the import error messages you forwarded are valid, caused by a "US
County" that is inconsistent with the "US State". Your QSO with KE4EK, for
example, specifies a state of FL and a county of KY,Fayette.

+ So the error message

"invalid secondary administrative subdivision: KY,Fayette"

+ is exactly right.

+ If you still encounter difficulties after eliminating the interference from your
anti-malware application, don't hesitate to let me know.

73,

Dave, AA6YQ





Dave AA6YQ
 

+ AA6YQ comments below

I agree that the security message is from my anti-malware program. However, the initial pop-up is from QRZ.com. I think it is responding to the Pathfinder request to download the information for the QSO

+ I have never seen QRZ.com cause a web browser to display a popup window, nor has such an occurrence ever been reported here.

73,

Dave, AA6YQ

 

To close this out:

Dave suggested I use the XML feature of QRZ. I thought I was, but when I looked, I had the wrong password for QRZ in the Callbook Tab of DXKeeper Config and had Pathfinder set up as a backup. On the Import, DXKeeper was using the Pathfinder interface which exceeded the rate that QRZ allows. The DXKeeper Error Log kept showing incorrect password on QRZ which is why I looked at the password in the config screen. I noticed on my QRZ page that I had made 219 HTML requests and no XML requests. Now that I have put the proper password on DXKeeper Callbook tab, everything should be good.

Embarrassing but hopefully fixed.

Bob AF9W

-----Original Message-----
From: DXLab@groups.io <DXLab@groups.io> On Behalf Of Dave AA6YQ
Sent: Sunday, January 12, 2020 12:56 PM
To: DXLab@groups.io
Subject: Re: [DXLab] Pathfinder "509 Too Many Requests" Importing Contest
QSO's

+ AA6YQ comments below

I agree that the security message is from my anti-malware program. However,
the initial pop-up is from QRZ.com. I think it is responding to the Pathfinder
request to download the information for the QSO

+ I have never seen QRZ.com cause a web browser to display a popup window,
nor has such an occurrence ever been reported here.

73,

Dave, AA6YQ


Dave AA6YQ
 

+ AA6YQ comments below

Dave suggested I use the XML feature of QRZ. I thought I was, but when I looked, I had the wrong password for QRZ in the Callbook Tab of DXKeeper Config and had Pathfinder set up as a backup. On the Import, DXKeeper was using the Pathfinder interface which exceeded the rate that QRZ allows.

+ If DXKeeper is configured to use "QRZ.com via Pathfinder" on the Configuration window's Callbook tab and you import QSOs with the "Query callbook and DXCC database for missing items" option enabled on the Main window's "Log QSOs" tab, then DXKeeper paces the rate at which QSOs are imported to "one every 4 seconds" to comply with QRZ management's request.

+ However, if you have DXKeeper configured to use "QRZ.com via Pathfinder" as the "Secondary Callbook", no "one every 4 seconds" pacing is imposed. With the "QRZ XML Data" lookup failing every time due to the incorrect password, "QRZ.com via Pathfinder" was being queried for each imported QSO, triggering QRZ's load limiter.

I'll correct this defect in the next version of DXKeeper.

73,

Dave, AA6YQ