|
Home > Archive > Unix administration > January 2004 > dump ufsdump via SSH works but.... sometimes core dumps
You are viewing an archived Text-only version of the thread.
To view this thread in it's original format and/or if you want to reply to
this thread please [click here]
| Author |
dump ufsdump via SSH works but.... sometimes core dumps
|
|
| uzi69mmm@yahoo.com 2004-01-23, 5:11 pm |
| Hello,
I have Sun Solaris 8 (2.8) with a SCSI 8mm tape drive that I'm
doing ufsdumps via ssh && dd to other Solaris boxes that works
just fine.
From the tapehost (Box that's has the drive attached)
I issue to dump:
ssh -l root <remote_host_name_other_sun_box> -n ufsdump 0fu -
<file_system_to_be_dumped> | dd bs=64k of=/dev/rmt/0n
To get the data back:
mt -f /dev/rmt/0 rew
mt -f /dev/rmt/0 fsf (if I dumped more than one dump to a single
tape....)
Then to restore the data:
dd bs=64k if=/dev/rmt/0n | ufsrestore vibf 64k -
WORKS just fine, speed is great! Like Sliced Bread baby!
Now I want to do the same thing only dump a Redhat 9.0 box
to the Sun box.
This works only for small test dumps, say just /etc or just /var:
From the tapehost (Box that's has the drive attached)
I issue
ssh -l root <remote_host_linux_box> /sbin/dump -0a -f - /var | dd
bs=64k of=/dev/rmt/0n
Then to read the tape as usual:
dd bs=64k if=/dev/rmt/0n | ufsrestore vibf 64k -
HOWEVER, when the dump is LARGE i.e say dumping root i.e. "/" I the
dump
is successful but the restore core dumps:
# dd bs=64k if=/dev/rmt/0n | ufsrestore vibf 64k -
Verify volume and initialize maps
Note: doing byte swapping
Dump date: Tue Jan 06 13:49:05 2004
Dumped from: the epoch
Level 0 dump of / (dir usr) on linuxbox:/dev/hda1
Label: /
Extract directories from tape
Segmentation Fault - core dumped
| |
|
|
| Steve Cousins 2004-01-23, 5:11 pm |
| Peter Bunclark wrote:quote:
>
< lots of garbage snipped >quote:
>
> So you're trying to do a Solaris ufsrestore on a Linux dump - do you
> expect that to work?
>
> Pete.
No, he was trying to dump from a Linux machine to a Sun tape drive and
then restore back to the Linux machine. Nothing wrong with the
concept. The problem is that dump for Linux does not work well. In
Linus Torvalds' words:
"Dump was a stupid program in the first place. Leave it behind."
You can find out more about his thoughts on dump at:
http://lwn.net/2001/0503/a/lt-dump.php3
I use Gnu tar to backup machines across the network and it works fine.
Good luck,
Steve
--
________________________________________
_____________________
Steve Cousins Email: cousins@umit.maine.edu
Research Associate Phone: (207) 581-4302
Ocean Modeling Group
School of Marine Sciences 208 Libby Hall,
University of Maine Orono ME 04469
| |
| Goran Larsson 2004-01-23, 5:11 pm |
| In article <3FFC4960.F89653A2@maine.edu>,
Steve Cousins <steve.cousins@maine.edu> wrote:
quote:
> No, he was trying to dump from a Linux machine to a Sun tape drive and
> then restore back to the Linux machine.
No. he did the dump on the Linux machine to a Sun tape drive and then
tried to use ufsrestore on the Sun to read the tape:
| # dd bs=64k if=/dev/rmt/0n | ufsrestore vibf 64k -
|
| Verify volume and initialize maps
| Note: doing byte swapping
| Dump date: Tue Jan 06 13:49:05 2004
| Dumped from: the epoch
| Level 0 dump of / (dir usr) on linuxbox:/dev/hda1
| Label: /
| Extract directories from tape
| Segmentation Fault - core dumped
He runs ufsrestore on something that has a different byte order than
the Linux machine (note the "Note: doing byte swapping" from ufsrestore)
and as restore is called restore in Linux and ufsrestore in Solaris he
certainly is not trying to do the restore on the Linux machine, most likely
on he is doing it on the Sun.
quote:
> Nothing wrong with the concept.
It will not work. The dump format is not portable between implementations.
quote:
> I use Gnu tar to backup machines across the network and it works fine.
I would never trust GNU tar to do any of my backups. It is seriously
broken and is not compatible with what the POSIX standard has to say
about tar.
--
Göran Larsson http://www.mitt-eget.com/
| |
| Steve Cousins 2004-01-23, 5:11 pm |
|
Goran Larsson wrote:quote:
>
> In article <3FFC4960.F89653A2@maine.edu>,
> Steve Cousins <steve.cousins@maine.edu> wrote:
>
>
> No. he did the dump on the Linux machine to a Sun tape drive and then
> tried to use ufsrestore on the Sun to read the tape:
You are correct! Sorry about that.
quote:
> | # dd bs=64k if=/dev/rmt/0n | ufsrestore vibf 64k -
As Goran says, this will not work. ufsrestoring a linux dump is not
going to work due to different file systems as well as endian problems.
quote:
> |
> | Verify volume and initialize maps
> | Note: doing byte swapping
> | Dump date: Tue Jan 06 13:49:05 2004
> | Dumped from: the epoch
> | Level 0 dump of / (dir usr) on linuxbox:/dev/hda1
> | Label: /
> | Extract directories from tape
> | Segmentation Fault - core dumped
>
> He runs ufsrestore on something that has a different byte order than
> the Linux machine (note the "Note: doing byte swapping" from ufsrestore)
> and as restore is called restore in Linux and ufsrestore in Solaris he
> certainly is not trying to do the restore on the Linux machine, most likely
> on he is doing it on the Sun.
>
>
> It will not work. The dump format is not portable between implementations.
As I (mis)read it the concept was ok. Of course, you can prove anything
if you start with a false premise :*)
quote:
>
> I would never trust GNU tar to do any of my backups. It is seriously
> broken and is not compatible with what the POSIX standard has to say
> about tar.
What do you use to backup linux machines? cpio? pax? There were a couple
of versions of Gnu tar that did have some problems but I've used Gnu tar
backups of some sort or another for many years with good results. I
primarily use it now because of the ability to do incremental backups,
which AFAIK can't be done with the non-Gnu tar.
Steve
________________________________________
_____________________
Steve Cousins Email: cousins@umit.maine.edu
Research Associate Phone: (207) 581-4302
Ocean Modeling Group
School of Marine Sciences 208 Libby Hall,
University of Maine Orono ME 04469
| |
| Xose Vazquez Perez 2004-01-23, 5:11 pm |
| Steve Cousins wrote:
quote:
> No, he was trying to dump from a Linux machine to a Sun tape drive and
> then restore back to the Linux machine. Nothing wrong with the
> concept. The problem is that dump for Linux does not work well. In
> Linus Torvalds' words:
>
> "Dump was a stupid program in the first place. Leave it behind."
>
> You can find out more about his thoughts on dump at:
>
> http://lwn.net/2001/0503/a/lt-dump.php3
>
> I use Gnu tar to backup machines across the network and it works fine.
There is more info about the dump 'bug' in: http://dump.sourceforge.net/isdumpdeprecated.html
| |
| Joerg Schilling 2004-01-23, 5:11 pm |
| In article <3FFC911D.B9C10372@maine.edu>,
Steve Cousins <steve.cousins@maine.edu> wrote:quote:
>
>
>As Goran says, this will not work. ufsrestoring a linux dump is not
>going to work due to different file systems as well as endian problems.
A real dump is able to deal with endian problems, there is a endian-structure
decoder in restore.
quote:
>
>As I (mis)read it the concept was ok. Of course, you can prove anything
>if you start with a false premise :*)
Many dump formats are portable, but note: the dump format is not a standard.
Anything may be changed without notice and cause problems.
quote:
>
>What do you use to backup linux machines? cpio? pax? There were a couple
>of versions of Gnu tar that did have some problems but I've used Gnu tar
>backups of some sort or another for many years with good results. I
>primarily use it now because of the ability to do incremental backups,
>which AFAIK can't be done with the non-Gnu tar.
GNU tar definitely cannot do incremental backups.
Many people believe that it can (it is even in the documentation), but
when you have a closer look at the basic idea and the implementation,
you will see that it has problems in design and implementation.
Even if it would work, it would cause a lot more tape to be written
than needed with ufsdump or star.
Star supports true incremental backups, it is not yet ready with its
incremental restore.
ftp://ftp.berlios.de/pub/star/alpha/
Star implements the same basic idea as ufsdump does. For this reason,
star has exactly the same problems (except for consistency).
The known problems with the incremental method used by star/ufsdump is that
it misses files that are created between incremenrtal backups where somebody
did set the mtime back to a value before the last full/incremental dump.
Star is based on the latest POSIX TAR standard and I hope that I will in future
be able to standardise the enhancemnts that are needeed to allow incremental
backups.
--
EMail:joerg@schily.isdn.cs.tu-berlin.de (home) Jörg Schilling D-13353 Berlin
js@cs.tu-berlin.de (uni) If you don't have iso-8859-1
schilling@fokus.fraunhofer.de (work) chars I am J"org Schilling
URL: http://www.fokus.fraunhofer.de/usr/schilling ftp://ftp.berlios.de/pub/schily
| |
| Goran Larsson 2004-01-23, 5:11 pm |
| In article <3FFC911D.B9C10372@maine.edu>,
Steve Cousins <steve.cousins@maine.edu> wrote:
quote:
> What do you use to backup linux machines?
Nothing. I don't backup any Linux machines, only Solaris machines.
The Solaris machines are backed up by my own backup system that
uses ufsdump inside.
quote:
> There were a couple
> of versions of Gnu tar that did have some problems but I've used Gnu tar
> backups of some sort or another for many years with good results.
You will find that it is likely only GNU tar that can read your
backups. That might be ok if you know about this and treat GNU tar as
having a proprietary format.
quote:
> I
> primarily use it now because of the ability to do incremental backups,
> which AFAIK can't be done with the non-Gnu tar.
The tar implementation "star" should be able to do what you require and
still create POSIX compliant backups.
--
Göran Larsson http://www.mitt-eget.com/
| |
| Joerg Schilling 2004-01-23, 5:11 pm |
| In article <3FFC9EF7.2000505@wanadoo.es>,
Xose Vazquez Perez <DELETExose@wanadoo.es> wrote:quote:
>Steve Cousins wrote:
>
>
>There is more info about the dump 'bug' in: http://dump.sourceforge.net/isdumpdeprecated.html
What these URLs describe are problems caused by an inconsistend Linux buffer
cache design. The real problems with the dump method are:
- general problems with life filesystems (not caused by the Linux design
problems), these can be solved by fssnap (avaliable since January 2001.
Either with Solaris 9 FCS or the Solaris 8 update that came out the same
time, FreeBSD introduced a similar thing at the same time).
- A nonstandard and not really portable archive format.
- Filesystem type specifics prevent you from using the same backup utility
for a bunch of different filesystems.
- It is just wrong to assume that an instance besides the kernel fs
implementation knows how to read the raw disk device.
It is a bad idea to access the raw disk for backup purposes.
- ufsdump is slow! It's antique design does not allow to have decent
bufering.
Star allows to virtually use any FIFO size. Defining e.g. 256 MB of FIFO
works with star and will give you more than 30 seconds tape streaming
reserve, even with modern fast tape drives.
- The ufsrestore interface ist just significantly worse and less flexible
than what you may do with star when it comes to restoring only a
specific choice of files.
- Ufsdump only works if you are root. It is impossible to use just one
utility/archive-format for every day data exchange and backups.
--
EMail:joerg@schily.isdn.cs.tu-berlin.de (home) Jörg Schilling D-13353 Berlin
js@cs.tu-berlin.de (uni) If you don't have iso-8859-1
schilling@fokus.fraunhofer.de (work) chars I am J"org Schilling
URL: http://www.fokus.fraunhofer.de/usr/schilling ftp://ftp.berlios.de/pub/schily
| |
| Joerg Schilling 2004-01-23, 5:11 pm |
| In article <Hr5As7.JvA@approve.se>, Goran Larsson <hoh@invalid.invalid> wrote:
quote:
>
>You will find that it is likely only GNU tar that can read your
>backups. That might be ok if you know about this and treat GNU tar as
>having a proprietary format.
Beware: Even GNU tar will not read back GNU tar dumps in many cases.
In case that you need to do multi-volume backups with GNU tar, GNU tar
implements a chance from 1-5% that it will not acept a continuation
volume. Star will be able to read those multi-volume GNU tar archives but
star does not implement GNU tars pseudo incremental backup system.
The reason is that the GNU tar incremental method does not work well and star
implements a better method.
--
EMail:joerg@schily.isdn.cs.tu-berlin.de (home) Jörg Schilling D-13353 Berlin
js@cs.tu-berlin.de (uni) If you don't have iso-8859-1
schilling@fokus.fraunhofer.de (work) chars I am J"org Schilling
URL: http://www.fokus.fraunhofer.de/usr/schilling ftp://ftp.berlios.de/pub/schily
| |
| Joerg Schilling 2004-01-23, 5:11 pm |
| In article <Hr5As7.JvA@approve.se>, Goran Larsson <hoh@invalid.invalid> wrote:quote:
>In article <3FFC911D.B9C10372@maine.edu>,
quote:
>
>The tar implementation "star" should be able to do what you require and
>still create POSIX compliant backups.
You may already do incremental backups using star (it implements the same basic
ideas as ufsdump does), but the incremental restore is not yet ready.
In case that files have been removed or renamed, you would need this specific
incremental restore feature. I expect about one man month of work until the
incremental restore is ready.
As I did rewrite about 2/3 of star during that last 6 months, I need a small
break and currently rather work on cdrtools.
--
EMail:joerg@schily.isdn.cs.tu-berlin.de (home) Jörg Schilling D-13353 Berlin
js@cs.tu-berlin.de (uni) If you don't have iso-8859-1
schilling@fokus.fraunhofer.de (work) chars I am J"org Schilling
URL: http://www.fokus.fraunhofer.de/usr/schilling ftp://ftp.berlios.de/pub/schily
| |
| uzi69mmm@yahoo.com 2004-01-23, 5:11 pm |
| > It will not work. The dump format is not portable between implementations.
Unless I missed something in my first post. The DUMP does work.
i.e. A sun ufsdump can "read" linux Dump. I've done AIX dump
and if you play with the correct blocks, ufsdump can read them...
| |
| Ed Murphy 2004-01-23, 5:11 pm |
| On Thu, 08 Jan 2004 01:01:49 +0000, Joerg Schilling wrote:
quote:
> - ufsdump is slow! It's antique design does not allow to have decent
> bufering.
>
> Star allows to virtually use any FIFO size. Defining e.g. 256 MB of FIFO
> works with star and will give you more than 30 seconds tape streaming
> reserve, even with modern fast tape drives.
>
> - The ufsrestore interface ist just significantly worse and less flexible
> than what you may do with star when it comes to restoring only a
> specific choice of files.
Disclaimer: Joerg is the author of star.
| |
| Steve Cousins 2004-01-23, 5:11 pm |
| Joerg Schilling wrote:quote:
>
> GNU tar definitely cannot do incremental backups.
What should I do with the GNU tar incremental tapes that I have been
backing up to and restoring from? I'm not sure what your reasoning is.
Maybe it isn't as efficient as dump would be (if dump worked
consistently), I don't know. But as someone who wants to be able to
back up just the files that have changed since the last full backup I
find that it works very well. Backups are quick and use just a small
amount of tape compared to the full backup. And of course I need to be
able to restore from these tapes, which I can. This fits my need for
incremental backups. I haven't tried to do multiple levels of
incremental backups and it seems that it can't do this (maybe I'm wrong
about this, I haven't looked into it). However, I've never found
multi-level ufsdumps to be that handy. I currently just do full and one
level of incrementals for our Sun machines. Restores are much simpler
this way.
I have GNU Tar on my IRIX and Tru64 machines too and I've not had
problems reading the Linux GNU Tar files. Maybe STAR would be the answer
to my dreams but GNU Tar does everything I need right now so I won't be
trying to fix something that isn't broken.
Regards,
Steve
________________________________________
_____________________
Steve Cousins Email: cousins@umit.maine.edu
Research Associate Phone: (207) 581-4302
Ocean Modeling Group
School of Marine Sciences 208 Libby Hall,
University of Maine Orono ME 04469
| |
| Joerg Schilling 2004-01-23, 5:11 pm |
| In article <3FFD73EB.9FA792E8@maine.edu>,
Steve Cousins <steve.cousins@maine.edu> wrote:quote:
>Joerg Schilling wrote:
>
>What should I do with the GNU tar incremental tapes that I have been
>backing up to and restoring from? I'm not sure what your reasoning is.
If you are _really_ happy with it....
quote:
>Maybe it isn't as efficient as dump would be (if dump worked
>consistently), I don't know. But as someone who wants to be able to
I forgot the exact background, but about 1-3 years ago, I did several tests
when I was planning the incremental backup method for star.
During this tests, it turned out, that there are cases that are not handled
correctly by GNU tar. You will not become aware of the problems until you
start to restore the incrementals from scratch.
Also note that if you rename a directory or chmod a large file, a GNU tar
incremental may be many gigabytes in size while a star or ufsdumo incremental
is only a few kilobytes.
quote:
>back up just the files that have changed since the last full backup I
>find that it works very well. Backups are quick and use just a small
>amount of tape compared to the full backup. And of course I need to be
The size that you observe seems to be a result of your usage profile.
A ufsdump or star incremental is typically much smaller.
quote:
>able to restore from these tapes, which I can. This fits my need for
Well, did you ever try to restore?
quote:
>incremental backups. I haven't tried to do multiple levels of
>incremental backups and it seems that it can't do this (maybe I'm wrong
GNU tar does not allow more than full + one incr. level which is bad.
Star allows any number of levels, I did limit the max level to 99
to make it simpler to print the numbers.
quote:
>about this, I haven't looked into it). However, I've never found
>multi-level ufsdumps to be that handy. I currently just do full and one
>level of incrementals for our Sun machines. Restores are much simpler
>this way.
A typical ufsdump usage is to have three different levels:
- Level 0 == full backup
- Level 1 done weekly
- Level 2 done daily you overwrite this tape after the next
weekly incremental has been made.
If you make level 0 dumps less frequent than once a month, you may like
to add another incremental level.
Please don't forget that there is a 1-5% chance that GNU tar does not
accept to read in a continuatuion volume from a multi _volume_ backup.
This is also something that you will never become aware unless you make
a restore test.
--
EMail:joerg@schily.isdn.cs.tu-berlin.de (home) Jörg Schilling D-13353 Berlin
js@cs.tu-berlin.de (uni) If you don't have iso-8859-1
schilling@fokus.fraunhofer.de (work) chars I am J"org Schilling
URL: http://www.fokus.fraunhofer.de/usr/schilling ftp://ftp.berlios.de/pub/schily
|
|
|
|
|