Topics

Back up storage

 

I am facing limits on both my active file and back up storage capabilities. I may very well exceed the limits on one or both within the next two months.

My last full base back up required almost 20 hours to complete 2T of data. This seems a bit unreasonable to me. While adding storage seems simple, the back up times present more problems. (Cloud storage appears to be too slow and expensive.) I have an open hard drive shelf in my Dell 8700. I can also swap my existing 4T for an 8T external drive. Installing either or both seems easy.

Can anyone provide advice for faster, yet dependable and secure, back up software?

Your thoughts or experience would be helpful.

Cheers,

David Clarkson

 

If you are on Mac - ChronoSync and ChronoAgent (http://www.econtechnologies.com/chronosync/overview.html). With Agent you can have automatic scheduling.

Your backup speed is also function of your drive connectivity. Having USB 3.0 drives or even Thunderbolt will likely reduce your backup time.
I currently have over 80TB of data at my house. Managing it tends to be challenging, especially when you have to move around data because you run out of storage ;-(
I would also recommend evaluating RAID system, so you protect your data against hardware failure.

Good luck … Oliver.

Oliver Klink
Art Ark Gallery, Artists’ Talk June 23, 2016 *NEW IMAGES*
June 2 - August 5, San Jose, California

*55Inches Exhibit - Abstraction Meets Reality Exhibit
Saratoga Library, Saratoga, CA
July 6 - September 5, 2016

*Festival de La Luz - Grand Opening of Consequences
Buenos Aires, Argentina
August 8 - September 30, 2016


On Jun 20, 2016, at 8:28 PM, David Clarkson <david@...> wrote:

I am facing limits on both my active file and back up storage capabilities. I may very well exceed the limits on one or both within the next two months.

My last full base back up required almost 20 hours to complete 2T of data. This seems a bit unreasonable to me. While adding storage seems simple, the back up times present more problems. (Cloud storage appears to be too slow and expensive.) I have an open hard drive shelf in my Dell 8700. I can also swap my existing 4T for an 8T external drive. Installing either or both seems easy.

Can anyone provide advice for faster, yet dependable and secure, back up software?

Your thoughts or experience would be helpful.

Cheers,

David Clarkson


 
Edited

As a (retired) certified Mac consultant, I agree with Oliver, and use Chronosync myself.

Using a backup that will only copy changed files, rather than re-copying -everything- will dramatically reduce backup time.

Based on my own experience, 2 TB backed up from scratch should take 8-10 hours; certainly not 20 hours. And keeping that updated with incremental (changed files only) would reduce the daily backup to about 20 minutes (depending on how much is changed, of course.)

Do be careful that your machine can handle 8TB drives. I looked up your machine, and see that it support SATA III, which is good, and if your drives are compatible, the speed should match what I mentioned above.

Tracy
www.valleau.gallery

On 20 Jun 2016, at 20:35, Oliver Klink wrote:

If you are on Mac - ChronoSync and ChronoAgent (http://www.econtechnologies.com/chronosync/overview.html). With Agent you can have automatic scheduling.
If you are on PC - Acronis (http://www.acronis.com/en-us/promotion/personal/SEM/?gclid=CK_pyomXuM0CFZOCfgodWtEINg)

Your backup speed is also function of your drive connectivity. Having USB 3.0 drives or even Thunderbolt will likely reduce your backup time.
I currently have over 80TB of data at my house. Managing it tends to be challenging, especially when you have to move around data because you run out of storage ;-(
I would also recommend evaluating RAID system, so you protect your data against hardware failure.

Good luck … Oliver.

Oliver Klink
*Wildlife Beyond Borders <http://www.oliverklinkphotography.com/Exhibits/WildlifeBeyondBorders> Exhibit
Art Ark Gallery, Artists’ Talk June 23, 2016 *NEW IMAGES*
June 2 - August 5, San Jose, California

*55Inches Exhibit <http://www.oliverklinkphotography.com/Exhibits/55Inches> - Abstraction Meets Reality Exhibit
Saratoga Library, Saratoga, CA
July 6 - September 5, 2016

*Festival de La Luz <http://www.encuentrosabiertos.com.ar/es/node/540> - Grand Opening of Consequences
Buenos Aires, Argentina
August 8 - September 30, 2016

@oliklink <mailto:oliver@...>
www.oliverklinkphotography.com <http://www.oliverklinkphotography.com/>
408-910-6701

On Jun 20, 2016, at 8:28 PM, David Clarkson <@DavidC> wrote:

I am facing limits on both my active file and back up storage capabilities. I may very well exceed the limits on one or both within the next two months.

My last full base back up required almost 20 hours to complete 2T of data. This seems a bit unreasonable to me. While adding storage seems simple, the back up times present more problems. (Cloud storage appears to be too slow and expensive.) I have an open hard drive shelf in my Dell 8700. I can also swap my existing 4T for an 8T external drive. Installing either or both seems easy.

Can anyone provide advice for faster, yet dependable and secure, back up software?

Your thoughts or experience would be helpful.

Cheers,

David Clarkson

--
Tracy Valleau, moderator

Imagemakers

www.valleau.gallery

 

On Mac try CarbonCopyCloner
On Windows try Acronis

Cloud storage can be very inexpensive and you have the huge advantage to have your files off-site in case of local disaster or theft.
Look at CrashPlan, plans start at $9 a month for unlimited storage. The initial upload will take a very long time but then it can backup in the background from then on. Retrieval can be done within the app of from their website.

Also, what is the current speeds and connections of your hard drives, that will make a difference.

On Jun 20, 2016, at 11:28 PM, David Clarkson <david@...> wrote:

I am facing limits on both my active file and back up storage capabilities. I may very well exceed the limits on one or both within the next two months.

My last full base back up required almost 20 hours to complete 2T of data. This seems a bit unreasonable to me. While adding storage seems simple, the back up times present more problems. (Cloud storage appears to be too slow and expensive.) I have an open hard drive shelf in my Dell 8700. I can also swap my existing 4T for an 8T external drive. Installing either or both seems easy.

Can anyone provide advice for faster, yet dependable and secure, back up software?

Your thoughts or experience would be helpful.

Cheers,

David Clarkson



 

I use Crashplan for cloud storage. As Mark said, the initial backup takes a long time...a really long time. But once that is done, additional backups are incremental. Having an offsite backup is very important in case of fire or theft.

Matt

On Jun 21, 2016, at 5:03 AM, Mark Savoia <mark@...> wrote:

On Mac try CarbonCopyCloner
On Windows try Acronis

Cloud storage can be very inexpensive and you have the huge advantage to have your files off-site in case of local disaster or theft.
Look at CrashPlan, plans start at $9 a month for unlimited storage. The initial upload will take a very long time but then it can backup in the background from then on. Retrieval can be done within the app of from their website.

Also, what is the current speeds and connections of your hard drives, that will make a difference.



On Jun 20, 2016, at 11:28 PM, David Clarkson <david@...> wrote:

I am facing limits on both my active file and back up storage capabilities. I may very well exceed the limits on one or both within the next two months.

My last full base back up required almost 20 hours to complete 2T of data. This seems a bit unreasonable to me. While adding storage seems simple, the back up times present more problems. (Cloud storage appears to be too slow and expensive.) I have an open hard drive shelf in my Dell 8700. I can also swap my existing 4T for an 8T external drive. Installing either or both seems easy.

Can anyone provide advice for faster, yet dependable and secure, back up software?

Your thoughts or experience would be helpful.

Cheers,

David Clarkson



--
Mark

stillrivereditions.com


--
Matt Connors

ImageMaker

www.inadvertentartist.com

 

On Crashplan, 4TB initial upload took about two weeks. That is with a upload service of 10Mbps from Comcast. But now I don’t even pay attention to it. Its working fine in the background, adding files as I do.

On Jun 21, 2016, at 9:29 AM, Matt Connors via Groups.io <mdconnors@...> wrote:

I use Crashplan for cloud storage. As Mark said, the initial backup takes a long time...a really long time. But once that is done, additional backups are incremental. Having an offsite backup is very important in case of fire or theft.

Matt



Michael Fryd
 

A good backup strategy should minimize single points of failure that can cause complete data loss.

Generally “Backups” are for the active drive attached to your computer, and “archives” are offline storage of older files.

A simple but effective backup strategy is to have two external backup drives (let’s call them “Blue” and “Red”).

The Blue drive is kept off-site. The Red Drive is attached to the computer and you run incremental backups every night. Once a week you swap the drives - take the Red drive to the off-site location and bring back the Blue Drive. The Red and Blue disks should be different brands.

Explanation:

Incremental backups are preferred rather than copying the entire disk. An incremental backup copies only those items that were supposed to have changed. A full disk image copies everything. If there is hidden disk corruption on your master drive, a full disk copy will overwrite the good files on your backup with the corrupted file. An incremental backup won’t copy a file if the OS doesn’t think it has changed.

An offsite and offline copy makes it difficult for malware to delete or encrypt the data. If malware encrypts all your online data, your offsite copy should still be OK.

The Red and Blue disks should be different brands. This reduces the likelihood that a single manufacturing defect will cause both backup drives to fail at the same time.

One backup should always be offsite. A common cause of data loss is someone stealing your computer and the attached drives. Others are a roof leak, knocking over the table, and a fire. An offsite copy helps protect against these hazards.

When swapping backup drives, you should always bring the local disk to the remote site, and then the other remote disk back. You want to avoid the situation where all copies of the data are in the same place.



As to RAID devices, these are helpful, but do not eliminate the need for other backups. A RAID like device can protect you from data loss due to a single hard drive failure, but there are still many hazards that can cause the loss of all data on the RAID device. For example: the failure of the RAID controller, user error overwriting your master file with a low rez web image, someone knocking over the table, a roof leak, or theft.


Obviously, one can get even more carried away. For instance, you can add additional disks to the backup rotation, or make archive copies of your projects on other types of media (perhaps optical media).



When it comes to archiving, I suggest a similar strategy. Keep multiple copies in multiple locations. Every few years, merge your archives onto newer (larger media). The capacity of storage devices is increasing and price is going down. After a few years you should be able to copy multiple backup archives onto a single new storage device.

Archiving has other issues which add an additional challenge. Simply saving a file may not be enough, you may not have software that can read it. I have archived images on PhotoCD discs. I don’t have software that can read these files (Photoshop dropped support long ago). I have digital images from one of the first consumer digital cameras (Apple’s QuickTake 100). I no longer have software than can open these files. Actually, I still have the software, I no longer have a computer that can run the software.


Of course, the above is not the only reasonable strategy. There are many reasonable backup strategies. Depending on how important you data is, you can make a minimal effort, or go way overboard. You have to determine what it’s worth to you to protect your data.

The important concept in any good backup strategy is that it minimizes the chances you will lose your data.

 

Michael

The on-site/off-site alternating drive strategy has worked for decades, I agree. The basic concept of keeping a copy of the data in a second location remains important.

I think, though, that the advent of cloud backup makes the alternating drive strategy less attractive. That strategy has a couple of drawbacks, e.g. where to keep the off-site drive and overall forgetfulness/laziness in making the switch. The possibility of losing up to a week’s info is still there. Those problems are reduced with cloud backup. You don’t have to remember to switch drives. The time period of data loss is reduced. Crashplan runs pretty much continuously, for example, looking for files to upload. In short, it’s easier, less prone to user error, and as reliable.

Btw, I’m not saying Crashplan is definitely the way to go. There may be a better option. But, look for one that monitors certain folders for changes and automatically backs up changed files in those folders.

Cheers,
Matt

On Jun 21, 2016, at 7:17 AM, Michael Fryd <yahoo2@...> wrote:

A good backup strategy should minimize single points of failure that can cause complete data loss.

Generally “Backups” are for the active drive attached to your computer, and “archives” are offline storage of older files.

A simple but effective backup strategy is to have two external backup drives (let’s call them “Blue” and “Red”).

The Blue drive is kept off-site. The Red Drive is attached to the computer and you run incremental backups every night. Once a week you swap the drives - take the Red drive to the off-site location and bring back the Blue Drive. The Red and Blue disks should be different brands.

Explanation:

Incremental backups are preferred rather than copying the entire disk. An incremental backup copies only those items that were supposed to have changed. A full disk image copies everything. If there is hidden disk corruption on your master drive, a full disk copy will overwrite the good files on your backup with the corrupted file. An incremental backup won’t copy a file if the OS doesn’t think it has changed.

An offsite and offline copy makes it difficult for malware to delete or encrypt the data. If malware encrypts all your online data, your offsite copy should still be OK.

The Red and Blue disks should be different brands. This reduces the likelihood that a single manufacturing defect will cause both backup drives to fail at the same time.

One backup should always be offsite. A common cause of data loss is someone stealing your computer and the attached drives. Others are a roof leak, knocking over the table, and a fire. An offsite copy helps protect against these hazards.

When swapping backup drives, you should always bring the local disk to the remote site, and then the other remote disk back. You want to avoid the situation where all copies of the data are in the same place.



As to RAID devices, these are helpful, but do not eliminate the need for other backups. A RAID like device can protect you from data loss due to a single hard drive failure, but there are still many hazards that can cause the loss of all data on the RAID device. For example: the failure of the RAID controller, user error overwriting your master file with a low rez web image, someone knocking over the table, a roof leak, or theft.


Obviously, one can get even more carried away. For instance, you can add additional disks to the backup rotation, or make archive copies of your projects on other types of media (perhaps optical media).



When it comes to archiving, I suggest a similar strategy. Keep multiple copies in multiple locations. Every few years, merge your archives onto newer (larger media). The capacity of storage devices is increasing and price is going down. After a few years you should be able to copy multiple backup archives onto a single new storage device.

Archiving has other issues which add an additional challenge. Simply saving a file may not be enough, you may not have software that can read it. I have archived images on PhotoCD discs. I don’t have software that can read these files (Photoshop dropped support long ago). I have digital images from one of the first consumer digital cameras (Apple’s QuickTake 100). I no longer have software than can open these files. Actually, I still have the software, I no longer have a computer that can run the software.


Of course, the above is not the only reasonable strategy. There are many reasonable backup strategies. Depending on how important you data is, you can make a minimal effort, or go way overboard. You have to determine what it’s worth to you to protect your data.

The important concept in any good backup strategy is that it minimizes the chances you will lose your data.



--
Matt Connors

ImageMaker

www.inadvertentartist.com

 

Mark's suggestion of cloud storage is a good one. There are lots to choose from. (I bought unlimited online storage from Amazon. I think it was $69 per year.)

One thing I neglected to mention regarding your drives is this: the more full a drive becomes, the slower writes will be. For a general running drive, leaving at least 10% free is wise. Trying to write data to drives that are basically full, even if you "delete a few files to make room," will be far slower than copying the same files to a drive that is, say, 50% full.
--
Tracy Valleau, moderator

Imagemakers

www.valleau.gallery

 

Thank you to everyone who responded. After a bit more work I signed up with Crash Plan. First backup underway now running both the desktop and laptop. If I learn anything of interest I will pass it along.
Cheers,
David

On Tue, Jun 21, 2016 at 9:36 AM Tracy Valleau <tracy@...> wrote:

Mark's suggestion of cloud storage is a good one. There are lots to choose from. (I bought unlimited online storage from Amazon. I think it was $69 per year.)

One thing I neglected to mention regarding your drives is this: the more full a drive becomes, the slower writes will be. For a general running drive, leaving at least 10% free is wise. Trying to write data to drives that are basically full, even if you "delete a few files to make room," will be far slower than copying the same files to a drive that is, say, 50% full.


--
Tracy Valleau, moderator

Imagemakers

www.valleau.gallery

Jim Kasson
 

I tried to use Crashplan. I have a full-duplex 50 Mb/s fiber ISP link. Initially, upload speeds were in the 5 Mb/s region, but dropped lower and lower as time went by. After several months, my 6 TB backup was about a third done. The the desktop client program stopped working, and I never put much effort into getting it started again because the performance was so bad.

Crashplan used to offer to ship you a hard disk with your data for recovery, but they no longer do so. At the rates I saw -- assuming they applied to downloads -- it would take almost a year to recover 6 TB of data. 

I'm back to carting spinning rust 8TB drives back and forth from the safe deposit box.

Jim
--
Jim Kasson

ImageMaker

Blog: blog.kasson.com

Gallery: www.kasson.com