Re: Import Enhancement request (Long ignore)


Dave AA6YQ
 

+ AA6YQ comments below

 

This started out as a bug report, and is ending up as an enhancement request....  I think.  I turn all the evidence and conclusions over to the experts for resoultion.

 

It starts out as a simple task of help  a newcomer to DXL up to speed. After doing all that, the user says " I have an old log on my Apple that I would like to import.

He gives me the name of the software  MacLoggerDX.

I look up the program on the www and find that it will export adif files, so I tell my friend sure, send me the file and we can determine what will and won't be imported as a result.

So he sends me the file and I have a look. Header says it is Adif version 3.0.3 

So I make a new blank mdb and import it.   Looks pretty good to me.

I tell my buddy it imported 561 QSO's .  He says there should be a lot more than that.

Hmmm   I didn't get any error messages... and it returned to the import page normally.  So I import the adif file into  ADIF master.  Sure enough  6211 QSO's.

So now the head scratching phase starts.  Since the import process appears to be aborting without reporting an error  I need to find out what is causing the issue.   So I sort the ADIF by callsign worked, export it as an adif  and import that.    Same issue, different place, but now by sorting the log by callsign, I can find where the mishap is ocuring.

First entry not imported is a qso with OK2D.     So I delete that record in the adi file  and try again.  Now it goes further, but aborts again . Same process  but this time it is S51MT  .  So once again I delete the record and import again. Success all 6000+ remaining records are imported along with the exception report for errors detected .

So now  why ....   looking at the adi with notepad at the two deleted records, an square characters is  in both records  .   I am pasting it here, who knows how that will translate.  In fact it did not let me paste it visibly into my mail program (Thunderbird).

Ok so time to figure out who is at fault and why.

Looking at the two callsigns  on qrz.com yields the clue.   My buddy uses the callsign lookup features both in DXL and  MacloggerDX, so he has name and address fields filled by qrz lookup for most of those QSO's.

Those two callsigns are using charchters note in the 437 codepage.   Now we are down to the why, on to the who.

Researching the ADIF specs reveals that it does allow for such characters. In ourder to use them, you should use the XML vesion of ADIF which results in an ADX extension file..  ok  MacloggeerDX can do that, so they have accounted for the situation. 

But DXK cannot import an ADX file, so that is not the solution.   Hmmm  what to try next.

I go back to qrz.com and look up ok2d ( I do not have an xml lookup account)  I created a qso record in DXK  and I copy the character in question ....    it is in the address:


756 11 Valašská Polanka, CZ

It came out properly here now. 

So I pasted into DXK and saved the record.  Works and displays fine.  So I export it as an adi. Look at it with notepad  ....  looks ok still.    Import it back in ...   works fine.     arrrgh

Since we have done the import ok, we let it go at that.

Several weeks later I am helpinghim again to do an import of some wsjt qso's that got lost when dxk was accidentally not running and he logged some qso's .  

It is now for the first time I notice the checkbox

[ ] Ignore tags with binary data

Aha, I never noticed that one before, and never would have either  except that is now likely what was happening on that import a few weeks ago,   So curious as ever, I nuke my test mdb. check the box  and import the original MacloggerDX  adi file..  Perfect. ....  no one was at fault after all, and both had solutions.

 

So in conclusion:

Maybe Maclogger created the ADI incorrectly  not sure, but being able to generate an ADX gets him off the hook per ADIF specs.

DXK is able to get around it with the checkbox,  but  who would find it with the description above ??  Not the average user I am afraid.

Solution :  Several possible..  I think the best might be to examine the adi file looking for tags that contain binary data  and if found, turn the checkbox function on without asking  or

add a button on the import type type list to include an entry for MacloggerDX  and turn it on as a result.

or can DXK import an ADX and I missed it ?

or ???

+ So that I can see what's going on, please attach the original 6211-QSO ADIF file to an email message, and send the message to me via aa6yq (at) ambersoft.com

+ The "Ignore tags with binary data" option is provided to deal with ADIF files that contain "binary data" (character code less than 32, or character code greater than 127) either in field names or field values.

+ Many years ago, several amateur radio software developers advocated for the abandonment of ADIF in favor of an XML-based interchange specification. Their arguments for this boiled down to "it would be cool". To avoid a complete fork, I advocated for extending ADIF with an XML-based syntax (called ADX) that references the same fields defined in ADIF, and employs unicode (UTF-8) in international character strings. This proposal was adopted by majority vote, and the ADIF specification was duly expanded to specify the ADX syntax as well as the original ADI syntax.

+ However, since having a "cool" interchange specification has little appeal to users, there is been very little adoption of ADX. If it ever gains traction, I will extend DXKeeper to support it.

            73,

                   Dave, AA6YQ

 

Join DXLab@groups.io to automatically receive all group messages.