|
Home > Archive > Backup Software > October 2004 > Best way to backup and synchronize 30,000 files
You are viewing an archived Text-only version of the thread.
To view this thread in it's original format and/or if you want to reply to
this thread please [click here]
| Author |
Best way to backup and synchronize 30,000 files
|
|
|
| The company I work for scan a lot of documents and store in tiff
format. Each file is about 100k to 1 meg in size. Total number of
files is arround 30,000 Which is about 20 Giga bytes.
Everyday, about 500 new files with temporary name will be added, 500
files will be renamed and 500 to 1000 files will be moved to another
directory.
My job is to mirror or synchronize all these new,renamed,relocated
files to another PC daily. And perform an incremental backup to DVD
using NERO BackItUp weekly.
My problems are
1. Because of the large number of files (30,000) I can't find a
synchronize software that is fast enough to finish the job within 1-2
hours
2. During the synchronization the 1500 renamed or relocated files will
be reported as NOT in source, I will delect these files on the
destination PC in order to prevent duplicated files. I am always
worry about what if someone accidentally delected a directory on the
source PC. I will delected the same files on the destination PC during
synchronize since they will be reported NOT in source also. I really
don't have time to verify whether a file is renamed, relocated or
delected when it is reported not in source.
3. Now I have 80 pieces of DVD Backup DATA which is over 320Giga
Bytes. Most of the files backed up have been renamed or moved. How am
I supposed to restore the data back? If both PC broke down tonight.
I go to work everyday, thinking for a better way to do the job, any
suggestion?
PS. Due to tight budget buying expensive hardware or software will not
be an option.
| |
| Toshi1873 2004-10-28, 5:45 pm |
| In article <f86c2831.0410260843.3b1ea914
@posting.google.com>, ktsps66@yahoo.com.hk says...
> The company I work for scan a lot of documents and store in tiff
> format. Each file is about 100k to 1 meg in size. Total number of
> files is arround 30,000 Which is about 20 Giga bytes.
> Everyday, about 500 new files with temporary name will be added, 500
> files will be renamed and 500 to 1000 files will be moved to another
> directory.
> My job is to mirror or synchronize all these new,renamed,relocated
> files to another PC daily. And perform an incremental backup to DVD
> using NERO BackItUp weekly.
> My problems are
>
> 1. Because of the large number of files (30,000) I can't find a
> synchronize software that is fast enough to finish the job within 1-2
> hours
Tried Second Copy 2000 or even rsync? 30k files is
nothing compared to what I backup daily with both of
those tools.
>
> 2. During the synchronization the 1500 renamed or relocated files will
> be reported as NOT in source, I will delect these files on the
> destination PC in order to prevent duplicated files. I am always
> worry about what if someone accidentally delected a directory on the
> source PC. I will delected the same files on the destination PC during
> synchronize since they will be reported NOT in source also. I really
> don't have time to verify whether a file is renamed, relocated or
> delected when it is reported not in source.
Both Second Copy and rsync offer the option to have both
an "active" backup target and a "trash" backup target.
Any files that change or are deleted are moved to the
"trash" tree prior to moving the latest version of the
file to the "active" tree. (Second Copy allows you to
adjust the number of old versions to keep.) The
"active" tree is always an exact copy of the source tree
once the backup finishes.
> 3. Now I have 80 pieces of DVD Backup DATA which is over 320Giga
> Bytes. Most of the files backed up have been renamed or moved. How am
> I supposed to restore the data back? If both PC broke down tonight.
Use DVD for archival and worst-case restoration. Use
disc cataloguing software (i.e. SuperCat) to help
tracking down copies of files that need to be restored.
A better option for disaster recovery would be removable
drives (tape would be ideal, but has a very hefty up-
front cost, removable discs can be done piecemeal). A
removable 160GB drive should run about $150 for the
drive + enclosure/bay. With a few removable drives, you
could snapshot both machines daily, rotating the drives
off-site once per week.
|
|
|
|
|