Backup to Tape

[Werby]

I'm wondering what folks are using to back-up their servers. We currently have about 10TB of live data that we back up nightly to LTO3 tapes using Retrospect 6.5 running on an xserve.

The problem is that this won't scale too well as our server expands. Right now it takes about 40 tapes to do the initial backup to tape, which takes about 4 days. The subsequent incremental backups that occur nightly are roughly 100 GB so I can get 4 days on a tape. My library fills up after about a month and I have to start over, thus missing up to 4 days of incremental backups.

Retrospect 6.5 is no longer supported by EMC, and version 8 runs 10 TIMES slower! EMC claims they are working on the problem but 8 has been out for over a year and it still doesn't work. I need to switch to a currently supported solution that can back up 10TB in 48 hours or less. I'm going to upgrade my tapes and drives to LTO4, but I still need a software solution that costs less than $1000. Any ideas?

-Todd Shirley

[sgirard]

40 tapes seems like too many for 10TB. IIRC, LTO3 is 400GB uncompressed and 800 with a good compression ratio.

Some other things to think about: Can you split off some of that storage into an archive that uses a different (separate) backup strategy? Some of that 10TB is probably not changing weekly or monthly or ever again.

If you are up against hardware/time/physical limitations maybe you could look at adding a second library and do your full backups to one and your diffs to another. Or, along the same lines put in a disk-to-disk backup over your network, diff it nightly or whenever and just put your weekly fulls on tape.

[PrePress Guru]

Your biggest bottleneck is usually the IO in the computer and not the tape drive/library.
I had/have the same problem with our LTO1 and LTO2 drives, G4 & G5 servers and a Raid 5 scsi array.
They are rated for more throughput then the computers IO and raid array can handle.
Its the same with scsi, the potential throughput is huge but the actual speeds are low.

OSX has few good options for enterprise level backups. Retrospect 6.5 is still the only one that seems to work.
If you are willing to buy into a PC, you would have a ton of great options.
Retrospect PC, ArcServe, Backup Exec, Arkeia....Ive used a few of them and they are complicated but great.
With a gigabit or 10gbe connection you would be able to backup massive quantities of data and scale up as needed.

As sgirard said, you should determine if you really need all 10TB backed up that often.
If it doesn't change, archive and remove it from your daily scripts.

I hope this helps, Good luck!

[cmcfarling]

Our setup is a little less than half your size. We're backing everything up to disk now. It's much more seamless and fewer tape related headaches to deal with. We run Retrospect 7.6 on a 7 year old Dell PowerEdge 1650. Backing up OSX machines is a non-issue with Retrospect Client software. Backing up to disk removes the concept of full and incremental backups, which is nice. Instead you tell Retrospect to use X amount of disk space for each storage set. Once that fills up it grooms the data removing old data to make room for new. Instead of having a fixed one or two weeks of backup data and then a recycle backup, you can have several weeks of backup data. Of course it all depends on the size of your backup storage. We have 7.5TB for backup storage.

The down side... Retrospect's grooming feature is less than perfect. Automatic grooming fails much more than it should. It tends to be with backup sets that store large amounts of data, which would be the case for you it sounds like. The usual fix is rebuilding the backup set, which can takes hours and hours. On the bright side the backup data remains in tact, it's just the backup set that gets corrupted. I would encourage you to browse through the Retrospect forum and search for grooming related issues if your interested in going this route.

Overall backing up to disk with Retrospect is a definite improvement over tape, for us at least. File retrieval can be done in literally seconds in some cases, and remotely at that.

[otherthoughts]

Perhaps this thread might add somewhat to the discussion?
Back up And Archiving ....ideas

Regards -OT

[Werby]

Thanks to everyone for their input! Sgirard is correct that 40 tapes is too much for 10TB of data... I made a typo, meant to say 16TB! Most of our data is images and they just don't compress very well using LTO hardware compression; we usually only get about 450GB per tape.

We do archive off data on a regular basis, usually about 450GB per month, but that just breaks even with the amount of new data we create. While it's true that most of the data on the server isn't ever going to change, it is vastly helpful to have instant access to past years work. Our clients constantly request that we pick up old images, and every single time I run an archive there is almost immediately a request for something from it!

Does anyone have any experience with BRU Server? BRU Server Product Information - TOLIS Group, Inc. - The Backup and Recovery Experts
We are considering replacing Retrospect with this so I am trying to find out more about it. What are the pros and cons?

Regarding backup to disk - it just seems prohibitively expensive, especially since we are on the verge of having 20TB of live storage. We have grown accustomed to having access to any particular day's backup going back about 3 months, at which point we recycle the tapes. I'm open to all suggestions though, and I love hearing what other people are doing, so keep it up!

[cmcfarling]

Regarding backup to disk - it just seems prohibitively expensive, especially since we are on the verge of having 20TB of live storage.

While it's true you could spend a huge amount on a disk backup system, you could also do it relatively inexpensively. Three years ago our expenditure included
12 750GB drives ($360)
1 Promise enclosure ($2200)
1 copy of Retrospect Multi Server ($650)

You can get 2TB drives now for $200 or less. For the amount of data you're talking about I would consider a more robust software package than Retrospect though, although I don't know what that would be.