|
Home > Archive > Data Storage > March 2006 > Client/Server Backup Software for Linux with Tape-Library support
You are viewing an archived Text-only version of the thread.
To view this thread in it's original format and/or if you want to reply to
this thread please [click here]
| Author |
Client/Server Backup Software for Linux with Tape-Library support
|
|
| Emanuel Ziegler 2006-03-25, 10:01 am |
| Hi all!
Since the amount of data to backup in our institute is several TB, I was
looking for a tape library like the Overland Neo 2000 or
Quantum PX502 with LTO 3 tapes (12 TB native) and automatic changer (U320
SCSI). Everything should run on Linux.
The problem is the following. The library consists of 30 tapes at 400 GB
each. There are two kinds of filesystems to backup:
- A local filesystem of 10 TB size which should get a full backup
about once per 2 weeks (I'll call id "RAID")
- Several remote filesystems of about 10 GB each which have to be
backuped every day with incremental backups that are able to recover
every state of the last 2 week at least (I'll call it "HOMES")
The latter one (HOMES) was done by amanda until now with a DLT tape drive
and manual changing. Since amanda also allows changing scripts it should
be able to do the job, but the RAID spoils it.
In order to do all backup jobs with the same tape library I have the
following alternative solutions in mind:
- Doing a daily backup (one tape per day) with parts of the RAID and
all the HOMES such that I have a full backup of all the files on
RAID where no file is older than two weeks
Problems: Amanda can not do partly backups
- Dividing the tape library into 25 tapes for RAID and 5 tapes for
HOMES - Every two weeks the whole RAID is backuped while the HOMES
get backuped daily
Problems: Amanda can not stripe a FS over multiple tapes, the tapes
for HOMES are way too large such that they have to be reused without
overwriting the data on them (like dividing every tape into 5
sections). This is not easy with amanda.
Alternatives would be dar (which can span a FS over multiple media, but is
not designed for tape libraries) in combination with amanda and some
additional scripting. But I'd prefer a all-in-one solution.
So the backup software I'm looking for should provide:
- Client/Server architecture for remote FS
- Support for tape changer
- Striping of FS over multiple tapes or partly backups
- Incremental backups
- Ability to reuse tapes without overwriting them
Any suggestions are welcome. OpenSource software is preferred, but
commercial products up to $1000 are also possible.
Thanks,
Emanuel
| |
| Joshua Baker-LePain 2006-03-25, 10:01 am |
| On 2006-03-24, Emanuel Ziegler <eziegler@web.de> wrote:
>
> The problem is the following. The library consists of 30 tapes at 400 GB
> each. There are two kinds of filesystems to backup:
> - A local filesystem of 10 TB size which should get a full backup
> about once per 2 weeks (I'll call id "RAID")
> - Several remote filesystems of about 10 GB each which have to be
> backuped every day with incremental backups that are able to recover
> every state of the last 2 week at least (I'll call it "HOMES")
>
> The latter one (HOMES) was done by amanda until now with a DLT tape drive
> and manual changing. Since amanda also allows changing scripts it should
> be able to do the job, but the RAID spoils it.
Err, why is that? I use amanda and an LTO3 Neo2K changer to backup
a 5.5TB RAID, and I'll soon be adding another 6TB RAID.
> In order to do all backup jobs with the same tape library I have the
> following alternative solutions in mind:
> - Doing a daily backup (one tape per day) with parts of the RAID and
> all the HOMES such that I have a full backup of all the files on
> RAID where no file is older than two weeks
> Problems: Amanda can not do partly backups
Not true. If you use tar as your backup program rather than dump, amanda
is quite happy to do backups of chunks of a filesystem rather than the
whole filesystem. This is the solution I'd go with if I were you.
> - Dividing the tape library into 25 tapes for RAID and 5 tapes for
> HOMES - Every two weeks the whole RAID is backuped while the HOMES
> get backuped daily
> Problems: Amanda can not stripe a FS over multiple tapes, the tapes
Not true as of amanda 2.5.0, released yesterday (and in testing for a long
time).
> So the backup software I'm looking for should provide:
> - Client/Server architecture for remote FS
> - Support for tape changer
> - Striping of FS over multiple tapes or partly backups
> - Incremental backups
> - Ability to reuse tapes without overwriting them
>
> Any suggestions are welcome. OpenSource software is preferred, but
> commercial products up to $1000 are also possible.
Some former amanda users here have migrated to bacula, so you may want
to have a look at that. I did, but stuck with amanda because I found
bacula's scheduling facilities pracitcally non-existent, especially
compared to amanda's. So, again, if I were you, I'd simply divvy up the
RAID into multiple disklist entries and back it and the HOMES up using
the same amanda config.
--
Joshua Baker-LePain
Department of Biomedical Engineering
Duke University
| |
| Emanuel Ziegler 2006-03-25, 10:01 am |
| On Fri, 24 Mar 2006 12:55:06 +0000 Joshua Baker-LePain wrote:
>
> Not true. If you use tar as your backup program rather than dump, amanda
> is quite happy to do backups of chunks of a filesystem rather than the
> whole filesystem. This is the solution I'd go with if I were you.
Mhmm, is it possible to tell amanda/tar not to backup directorywise but
let's say 5% of the amount of data?
>
> Not true as of amanda 2.5.0, released yesterday (and in testing for a long
> time).
I was referring to the FAQ from the amanda page where the opposite was
claimed. Good to know that this information is deprecated.
> Some former amanda users here have migrated to bacula, so you may want
> to have a look at that.
I'll have a look at that as well. But the easiest would be to keep on
using amanda since it seems to do the task in the newest version.
Thanks,
Emanuel
| |
| Joshua Baker-LePain 2006-03-25, 10:01 am |
| On 2006-03-24, Emanuel Ziegler <eziegler@web.de> wrote:
> On Fri, 24 Mar 2006 12:55:06 +0000 Joshua Baker-LePain wrote:
>
> Mhmm, is it possible to tell amanda/tar not to backup directorywise but
> let's say 5% of the amount of data?
No, it's by directories, plus/minus include and exclude lists.
>
> I was referring to the FAQ from the amanda page where the opposite was
> claimed. Good to know that this information is deprecated.
The mailing list and new wiki are a wealth of information.
Good luck.
--
Joshua Baker-LePain
Department of Biomedical Engineering
Duke University
|
|
|
|
|