Import - multiple copes of media?


Sally Lloyd
 

I'm a new FH user.  A couple of weeks ago I imported a GEDCOM from RootsMagic. Looking at Media in FH I see I have multiple copies of some Media files, one for each associated citation. In RM I have 3,000+ files linked to this tree, in FH I have 15,000+. 

I must have done something wrong during in the import process, but what?
Many thanks, Sally


Mike Tate
 

Welcome Sally,

You have done nothing wrong except expose an annoying feature of FH that has been reported to Calico Pie but so far ignored.

I believe RootsMagic uses local Media attached to each Citation which FH converts to Media records on import.

However, FH does not recognise duplicates and creates a separate Media record and file for each copy.

You would have to manually merge the copies or somebody needs to write a Plugin to automate that process.

I suggest you complain to Calico Pie via their Support Centre ticket system and let us know what they say.

http://www.calico-pie.com/osticket/open.php

 

Mike Tate

 

From: family-historian@groups.io <family-historian@groups.io> On Behalf Of Sally Lloyd
Sent: 06 March 2021 09:14
To: family-historian@groups.io
Subject: [family-historian] Import - multiple copes of media?

 

I'm a new FH user.  A couple of weeks ago I imported a GEDCOM from RootsMagic. Looking at Media in FH I see I have multiple copies of some Media files, one for each associated citation. In RM I have 3,000+ files linked to this tree, in FH I have 15,000+. 

I must have done something wrong during in the import process, but what?
Many thanks, Sally


Sally Lloyd
 

Thank you Mike, I must say your explanation really surprised me!   A couple more questions. When the Project is backed up will I get another copy of the media folder? If so it's going to quickly fill up my hard drive. If I imported my tree into another program and then used that to make the Gedcom file would that help?   I would really like to be able to see all the links to an image in one place. 
Like Heather I was thinking of changing from a citation lumper into a citation splitter but I'm beginning to wonder if it's just going to be too much work on top of trying to sort out the media problem and whether I should choose another program altogether. 

I'll certainly report to support, thanks for the link.
Thanks again for your help,
Sally


Victor Markham
 

Sally

When you back up a project folder it is saved on your hard drive. The more timesyou back this up they are all saved and this fills ylour hard drive so I would delete the earlier back ups. I normally just keep one previopis back up

Victor

On 07/03/2021 9:11 am, Sally Lloyd wrote:
Thank you Mike, I must say your explanation really surprised me!   A couple more questions. When the Project is backed up will I get another copy of the media folder? If so it's going to quickly fill up my hard drive. If I imported my tree into another program and then used that to make the Gedcom file would that help?   I would really like to be able to see all the links to an image in one place. 
Like Heather I was thinking of changing from a citation lumper into a citation splitter but I'm beginning to wonder if it's just going to be too much work on top of trying to sort out the media problem and whether I should choose another program altogether. 

I'll certainly report to support, thanks for the link.
Thanks again for your help,
Sally


Sally Lloyd
 

Thank you Victor, that's good advice.

Sally


Sally Lloyd
 

Mike,

I've now imported a Gedcom file from Gramps and that seems to have worked as far as media and associated links are concerned. It's really not ideal having to import via another FH program, things get lost or changed on the way. This tree has moved more than enough times already,  it started in a small free program then it's travels went > PAF > Ultimate FT > TMG > Genbox > several versions of FTM > RM  and now to FH via Gramps!  Sometimes I peer into the outer reaches, groan and quickly close them.
I might try importing via FTM next although I haven't got the latest version of that.
Sally


Mike Tate
 

Sally,

It is difficult to be absolutely certain without access to your PC but I believe my explanation is correct.

Double-check with Calico Pie when you report the problem and supply as much detail as possible.

If possible, include snippets of GEDCOM before import showing where the Media (OBJE) are attached.

 

Regarding backup, it depends on what strategy you adopt.

If you use the FH Project Backup then yes Media files are included. That is sensible as they are very precious and some cost you money.

But there are backup options for just the GEDCOM and FH keeps Snapshots of the GEDCOM anyway.

However, you may have a PC wide backup regime for all Documents that would automatically include the FH Project and Media.

You may also synchronize with cloud storage such as OneDrive, Google Drive or DropBox.

Anyway, keeping copies of backups on the same hard drive gives false security because if the PC or drive crashes you have no backups.

You may know this already but it is important that backups are kept on some separate external media.

See FHUG Knowledge Base ‘Backup and Recovery’

https://fhug.org.uk/kb/kb-article/backup-and-recovery/

 

Many genealogy products prefer using local Media objects rather than Media records.

So migrating via another product is unlikely to help.

 

If Calico Pie cannot fix the problem, then a Plugin could be written to merge the multiple copies.

Such a Plugin would be needed by anyone importing from a variety of products, so would be well worth writing.

There is already the ‘Check for Possible Duplicate Media’ plugin that reports duplicates, which could be extended to auto-merge duplicates.

Try running that Plugin to confirm you have duplicates and include a summary of the results in the report to Calico Pie.

 

Mike Tate

 


John
 

Hi Sally & Victor,

Project backups don't have to go to the hard drive do they? I send my back-ups to a USB stick by default. I keep 3 generations, so you can tell how old I am, as that is how I was taught on main-frames!!

John


From: family-historian@groups.io <family-historian@groups.io> on behalf of Victor Markham via groups.io <victor@...>
Sent: 07 March 2021 09:17
To: family-historian@groups.io <family-historian@groups.io>
Subject: Re: [family-historian] Import - multiple copes of media?
 

Sally

When you back up a project folder it is saved on your hard drive. The more timesyou back this up they are all saved and this fills ylour hard drive so I would delete the earlier back ups. I normally just keep one previopis back up

Victor

On 07/03/2021 9:11 am, Sally Lloyd wrote:
Thank you Mike, I must say your explanation really surprised me!   A couple more questions. When the Project is backed up will I get another copy of the media folder? If so it's going to quickly fill up my hard drive. If I imported my tree into another program and then used that to make the Gedcom file would that help?   I would really like to be able to see all the links to an image in one place. 
Like Heather I was thinking of changing from a citation lumper into a citation splitter but I'm beginning to wonder if it's just going to be too much work on top of trying to sort out the media problem and whether I should choose another program altogether. 

I'll certainly report to support, thanks for the link.
Thanks again for your help,
Sally


Peter Atkinson
 

When you keep ''3 generations'' it is of course computer speak for ''3 files'' not generations as in family/people lines.
Have to remember some are not totally computer literate.
Peter.

On Mon, 8 Mar 2021, 09:42 John, <johngunson@...> wrote:
Hi Sally & Victor,

Project backups don't have to go to the hard drive do they? I send my back-ups to a USB stick by default. I keep 3 generations, so you can tell how old I am, as that is how I was taught on main-frames!!

John

From: family-historian@groups.io <family-historian@groups.io> on behalf of Victor Markham via groups.io <victor=markham.me.uk@groups.io>
Sent: 07 March 2021 09:17
To: family-historian@groups.io <family-historian@groups.io>
Subject: Re: [family-historian] Import - multiple copes of media?
 

Sally

When you back up a project folder it is saved on your hard drive. The more timesyou back this up they are all saved and this fills ylour hard drive so I would delete the earlier back ups. I normally just keep one previopis back up

Victor

On 07/03/2021 9:11 am, Sally Lloyd wrote:
Thank you Mike, I must say your explanation really surprised me!   A couple more questions. When the Project is backed up will I get another copy of the media folder? If so it's going to quickly fill up my hard drive. If I imported my tree into another program and then used that to make the Gedcom file would that help?   I would really like to be able to see all the links to an image in one place. 
Like Heather I was thinking of changing from a citation lumper into a citation splitter but I'm beginning to wonder if it's just going to be too much work on top of trying to sort out the media problem and whether I should choose another program altogether. 

I'll certainly report to support, thanks for the link.
Thanks again for your help,
Sally


David Hodgson-Brown
 

A lot of cloud backups allow versioning of files which allows you to go back to any date /time so you don't need to keep multiple copies. 


From: family-historian@groups.io <family-historian@groups.io> on behalf of Peter Atkinson <patkinson1943@...>
Sent: Monday, March 8, 2021 9:48:50 AM
To: family-historian@groups.io <family-historian@groups.io>
Subject: Re: [family-historian] Import - multiple copes of media?
 
When you keep ''3 generations'' it is of course computer speak for ''3 files'' not generations as in family/people lines.
Have to remember some are not totally computer literate.
Peter.

On Mon, 8 Mar 2021, 09:42 John, <johngunson@...> wrote:
Hi Sally & Victor,

Project backups don't have to go to the hard drive do they? I send my back-ups to a USB stick by default. I keep 3 generations, so you can tell how old I am, as that is how I was taught on main-frames!!

John

From: family-historian@groups.io <family-historian@groups.io> on behalf of Victor Markham via groups.io <victor=markham.me.uk@groups.io>
Sent: 07 March 2021 09:17
To: family-historian@groups.io <family-historian@groups.io>
Subject: Re: [family-historian] Import - multiple copes of media?
 

Sally

When you back up a project folder it is saved on your hard drive. The more timesyou back this up they are all saved and this fills ylour hard drive so I would delete the earlier back ups. I normally just keep one previopis back up

Victor

On 07/03/2021 9:11 am, Sally Lloyd wrote:
Thank you Mike, I must say your explanation really surprised me!   A couple more questions. When the Project is backed up will I get another copy of the media folder? If so it's going to quickly fill up my hard drive. If I imported my tree into another program and then used that to make the Gedcom file would that help?   I would really like to be able to see all the links to an image in one place. 
Like Heather I was thinking of changing from a citation lumper into a citation splitter but I'm beginning to wonder if it's just going to be too much work on top of trying to sort out the media problem and whether I should choose another program altogether. 

I'll certainly report to support, thanks for the link.
Thanks again for your help,
Sally


Victor Markham
 

Only 3 generations I have more than that!

Victor

On 8 Mar 2021, at 09:42, John <johngunson@...> wrote:
Hi Sally & Victor,

Project backups don't have to go to the hard drive do they? I send my back-ups to a USB stick by default. I keep 3 generations, so you can tell how old I am, as that is how I was taught on main-frames!!

John

From: family-historian@groups.io <family-historian@groups.io> on behalf of Victor Markham via groups.io <victor@...>
Sent: 07 March 2021 09:17
To: family-historian@groups.io <family-historian@groups.io>
Subject: Re: [family-historian] Import - multiple copes of media?
 

Sally

When you back up a project folder it is saved on your hard drive. The more timesyou back this up they are all saved and this fills ylour hard drive so I would delete the earlier back ups. I normally just keep one previopis back up

Victor

On 07/03/2021 9:11 am, Sally Lloyd wrote:
Thank you Mike, I must say your explanation really surprised me!   A couple more questions. When the Project is backed up will I get another copy of the media folder? If so it's going to quickly fill up my hard drive. If I imported my tree into another program and then used that to make the Gedcom file would that help?   I would really like to be able to see all the links to an image in one place. 
Like Heather I was thinking of changing from a citation lumper into a citation splitter but I'm beginning to wonder if it's just going to be too much work on top of trying to sort out the media problem and whether I should choose another program altogether. 

I'll certainly report to support, thanks for the link.
Thanks again for your help,
Sally


Leslie P
 

I have had contact with FH support (Calico Pie) regarding this issue, I have just been told that they consider it an active issue, they are currently working on it, they do understand the importance.

My note to them included the, hopefully gentle, request that they tell me if they hoped to have it fixed within the next six months, so I could determine whether to pursue a fix on my own, or wait for them. They said they hoped to have the fix within my time frame.

So...good news! Hang in there! It's going to get fixed.


Sally Lloyd
 

Many thanks Lesley. I too contacted customer support (8th March) and then again a couple of days ago. I had a nice reply from Martin in which he said much the same as you were told. I'd also asked if I could have a temporary license for FH6 so that I could use the plugin that sorts the problem out but they weren't able to do that.
Martin also thought that their solution would probably work on a new import rather than on an existing project (although he wasn't sure). This was a bit disappointing as I now have the option to either use the gedcom which came from RM7 via Gramps or wait in the hope someone comes up with a plugin or another solution.

This is an issue that must effect most people who move from RM to FH so I'm surprised that they haven't done something about it before?
Fingers crossed we don't have a long wait!!
Sally


On Thu, 25 Mar 2021 13:47 Leslie P, <leslie@...> wrote:
I have had contact with FH support (Calico Pie) regarding this issue, I have just been told that they consider it an active issue, they are currently working on it, they do understand the importance.

My note to them included the, hopefully gentle, request that they tell me if they hoped to have it fixed within the next six months, so I could determine whether to pursue a fix on my own, or wait for them. They said they hoped to have the fix within my time frame.

So...good news! Hang in there! It's going to get fixed.


Leslie P
 

You do bring up a good point - the problem they are working to fix is a problem of the import process, so yes, it would only be fixed for new imports not getting duplicate media.

Doesn't do a thing to fix folks who already have duplicated media and have already "done the final import" and are moving ahead.  That's going to need a plugin to merge all of the duplicate records. Hmmm.


Mike Tate
 

Yes, that is an interesting point.

There is the File > Validate command that could possibly perform the correction of duplicate media.

 

Local Media Objects (LMO) that RM and other products use can get added to a Project via File > Merge/Compare File and other scenarios.

Then the File > Validate command performs a similar conversion from LMO to Media Records as the GEDCOM import.

Hopefully, that will be fixed to avoid duplicates as well as the GEDCOM import process.

 

It would not be unreasonable to expect File > Validate to merge duplicate Media Records and files as part of its ‘validation’ regime.

Perhaps you could suggest that to Calico Pie since you have established a dialogue.

 

Mike Tate

 


Leslie P
 

Update - version 7.0.4.0 is the fix to the problem for new imports. I haven't tested it yet. That's GREAT news.

Now for the less great news:

"The fix only affects new imports. It doesn't correct the duplicate Media records in projects that have already been imported. We may create a plugin for this at some point, but I have no date for that if so."

That's from the email I got from FH support.


Sally Lloyd
 

Thanks Leslie,
I too had an email, yes great news!
I've now installed the update and its appears to work well with new imports. Although as you say it isn't going to help people who have already imported a gedcom, it won't work with existing projects.
Most impressed with how quickly they've sorted this out! 
Sally

On Tue, Apr 6, 2021 at 4:01 PM Leslie P <leslie@...> wrote:

Update - version 7.0.4.0 is the fix to the problem for new imports. I haven't tested it yet. That's GREAT news.

Now for the less great news:

"The fix only affects new imports. It doesn't correct the duplicate Media records in projects that have already been imported. We may create a plugin for this at some point, but I have no date for that if so."

That's from the email I got from FH support.


Mike Tate
 

We are looking at updating an existing Plugin to handle existing imported Projects.

Not sure about sorting it out quickly though. It was reported in July 2015 nearly 6 years ago.

I know because I reported it.

It is the repeated reports from new users that have prompted CP to belatedly get it fixed.

If it had been done anytime in the last 6 years then there would be no existing Projects needing a Plugin to correct them.

Rant over!

 


Leslie P
 

I was *this close* to going ahead with my "final import" and figuring I'd handle the de-duplication later. This is one instance where procrastination paid off!


Ron Chapman
 

I've tried to update the way recommended previously and the old way and it says I have the latest version 7.0.3.0!!

Ron


On 06/04/2021 21:37, Sally Lloyd wrote:
Thanks Leslie,
I too had an email, yes great news!
I've now installed the update and its appears to work well with new imports. Although as you say it isn't going to help people who have already imported a gedcom, it won't work with existing projects.
Most impressed with how quickly they've sorted this out! 
Sally

On Tue, Apr 6, 2021 at 4:01 PM Leslie P <leslie@...> wrote:

Update - version 7.0.4.0 is the fix to the problem for new imports. I haven't tested it yet. That's GREAT news.

Now for the less great news:

"The fix only affects new imports. It doesn't correct the duplicate Media records in projects that have already been imported. We may create a plugin for this at some point, but I have no date for that if so."

That's from the email I got from FH support.