Hi,

I'm not good at programming, but was wondering if there was a script
already created, or any howto on backing up.

This is what I want to do.

Basically there's a file server at a clients location.  I would like to
have selected directories and files(that I choose) to be backed up only
when changed (incremental).


then I would like it tared(or any other compression) and either put into
a folder named after todays date(or the tarred files name reflect the name).

anddddddd then, finally, needs to be sftp'ed into our server over the
internet.

and hopefully email us of the status of the transfer(I.E. when succesful
or if failed).


I hope someone can help me with this,  Thanks in advance.

I am thinking I need to setup a script with a cron job(like a few on the
net), and then run another cron job to run a program like logrotate(like
apache log rotator).  and then another cron job to email us.

I don't know,  I think I know the theory, just looking for a script or
an easier way to do this.


Thanks again in advance :-)

[ Post a follow-up to this message ]

In comp.os.linux.misc Gautam Bakshi <gautam.bakshi@gmail.com>:
> Hi,

> I'm not good at programming, but was wondering if there was a script
> already created, or any howto on backing up.

There are some nice backup howto (www.tldp.org) but AFAIK none of
them meets all your requirements.

> Basically there's a file server at a clients location.  I would like to
> have selected directories and files(that I choose) to be backed up only
> when changed (incremental).

> then I would like it tared(or any other compression) and either
> put into a folder named after todays date(or the tarred files
> name reflect the name).  anddddddd then, finally, needs to be
> sftp'ed into our server over the internet.  and hopefully email
> us of the status of the transfer(I.E. when succesful or if
> failed).  I hope someone can help me with this,  Thanks in
> advance.

> I am thinking I need to setup a script with a cron job(like a few on the
> net), and then run another cron job to run a program like logrotate(like
> apache log rotator).  and then another cron job to email us.

> I don't know,  I think I know the theory, just looking for a script or
> an easier way to do this.

Dunno about a ready available script meeting all your
requirements, try searching freshmeat.net.

You could pay someone near you developing and implementing stuff
for you, if you don't have the stuff competent enough to setup
something like this.

--
Michael Heiming (X-PGP-Sig > GPG-Key ID: EDD27B94)
mail: echo zvpunry@urvzvat.qr | PERL -pe 'y/a-z/n-za-m/'
#bofh excuse 398: Data for intranet got routed through the
extranet and landed on the internet.

[ Post a follow-up to this message ]

Gautam Bakshi <gautam.bakshi@gmail.com> wrote:
> I'm not good at programming, but was wondering if there was a script
> already created, or any howto on backing up.

Sure. Use tar or cpio. Get more elaborate as you like. Or use prebuilt
"solutions".

> Basically there's a file server at a clients location.  I would like to
> have selected directories and files(that I choose) to be backed up only
> when changed (incremental).

It's not clear what you mean here. I presume you want changed files
backed up as a whole, not the diffs backed up.

but

find `cat /mylist/of/files` -cnewer /last/backup/reference -fprint0 thislist
touch /last/backup/reference
tar czvfT --null - thislist | ssh server dd of=backup_`date +%Y%m%d`.tgz

Should do the trick, no? Elaborate to taste. I'd elaborate it into a
backup_server and a backup_client script. The latter on the server and
the former on the client, clearly!

Then from the server (backup_client) you'd run tar on the client
(backup_server).

> then I would like it tared(or any other compression) and either put into
> a folder named after todays date(or the tarred files name reflect the name).[/vbco
l]

What's the problem there?
[vbcol=seagreen]
> anddddddd then, finally, needs to be sftp'ed into our server over the
> internet.
>
> and hopefully email us of the status of the transfer(I.E. when succesful
> or if failed).

Do as you like.

> I hope someone can help me with this,  Thanks in advance.

But what's the problem?

> I am thinking I need to setup a script with a cron job(like a few on the

Sure - why not?

> net), and then run another cron job to run a program like logrotate(like

Why? What would it do? Is that at the server end?

> apache log rotator).  and then another cron job to email us.

Eh? What for?

> I don't know

Why don't you know. What has your thinking brought you?


> I think I know the theory


What "theory" would that be? I don't understand your use of the word.

> just looking for a script or

Then write one. Where's the difficulty? It's a few lines. Why don't you
just DO it then record your steps in the script? If you were to
actually show some effort people might well help you! As it is I don't
see why you don't just use one of the standard backup applications that
offer these sorts of things.

Peter

[ Post a follow-up to this message ]

In comp.os.linux.misc Peter T. Breuer <ptb@lab.it.uc3m.es>:
> Gautam Bakshi <gautam.bakshi@gmail.com> wrote: 
[..]
[vbcol=seagreen]
>    find `cat /mylist/of/files` -cnewer /last/backup/reference -fprint0 thi
slist
>    touch /last/backup/reference
>    tar czvfT --null - thislist | ssh server dd of=backup_`date +%Y%m%d`.tgz[/vbcol
]
[vbcol=seagreen]
> Should do the trick, no? Elaborate to taste. I'd elaborate it into a

Looks nice, I'd create some "backup" user, to ssh backup@server,
so you don't need to allow ssh root logins.;)

[..]

--
Michael Heiming (X-PGP-Sig > GPG-Key ID: EDD27B94)
mail: echo zvpunry@urvzvat.qr | PERL -pe 'y/a-z/n-za-m/'
#bofh excuse 291: Due to the CDA, we no longer have a root
account.

[ Post a follow-up to this message ]

Peter T. Breuer wrote:
> Gautam Bakshi <gautam.bakshi@gmail.com> wrote:
> 
>
>
> Sure. Use tar or cpio. Get more elaborate as you like. Or use prebuilt
> "solutions".
>

I have tried afbackup, amanda, kbackup, and varous other scripts.
(actually Kbackup is what's doing the back ups right now, then I have to
ssh into the clients fileserver, and download it into a directory that I
created on our server with the date)
> 
>
>
> It's not clear what you mean here. I presume you want changed files
> backed up as a whole, not the diffs backed up.
>
> but
>
>     find `cat /mylist/of/files` -cnewer /last/backup/reference -fprint0 th
islist
>     touch /last/backup/reference
>     tar czvfT --null - thislist | ssh server dd of=backup_`date +%Y%m%d`.t
gz
>
> Should do the trick, no? Elaborate to taste. I'd elaborate it into a
> backup_server and a backup_client script. The latter on the server and
> the former on the client, clearly!
>
> Then from the server (backup_client) you'd run tar on the client
> (backup_server).
>
> 
>
>
> What's the problem there?
>
> 
>
>
> Do as you like.
>
> 
>
>
> But what's the problem?
>
> 
>
>
> Sure - why not?
>
> 
>
>
> Why? What would it do? Is that at the server end?
>
well, by directly labeling the file above you kind of made this not
needed.  I was just going to do it, since it would automatically change
the name of the folder to the date and all new backups would go to
another folder.
> 
>
>
> Eh? What for?
>
> 
>
>
> Why don't you know. What has your thinking brought you?
>
>
> 
>
>
>
> What "theory" would that be? I don't understand your use of the word.
>
logic? is that a better word?  If that doesn't suit your needs, how
about your pick up?

> 
>
>
> Then write one. Where's the difficulty? It's a few lines. Why don't you
> just DO it then record your steps in the script? If you were to
> actually show some effort people might well help you! As it is I don't
> see why you don't just use one of the standard backup applications that
> offer these sorts of things.
Hey,  I didn't ask for you to do the work for me, if you have a problem
with people asking questions then why are you on newsgroups? and why did
you reply?  I had a problem, and wanted to know if anyone had any
experince with a simlair problem, or if my above "logic"(change word to
suit your own needs as needed) made sense.

if you can help, then help..if not or if your going to XXXXX then don't
bother answering people's questions. At the debian mailing list we get
litterally hundreds of posts a day, and no one there is rude, no matter
how basic the question or how many times asked.

So i didn't put any effort, tell me where to start then, please? any
books? websites?  I know how to back up, using tar(and now thanks to
you, i know how to back up incremental.  and also now thanks to you,
again,  I know how to name the backup file automatically.  I already
knew how to ssh into a machine without a username and password(saving
the server ssh key on the clients machine).  So what else do i need?

The one thing i can't figure out, is how to send the email actually,  I
normally know how to send emails when commands are successful from
command line, but i am not sure how to do it if it sftped corrected.  I
mean, if i do it based on $? (exit status) then i may get succesful
everytime since the script exits the ssh server correctly.  I don't know
how to script it once i'm sftped in.  How to exit the ssh server via a
script.

maybe i'll have to show some "effort" and practice it right?
>
> Peter

[ Post a follow-up to this message ]

Gautam Bakshi <gautam.bakshi@gmail.com> wrote: 
>
> So i didn't put any effort, tell me where to start then, please? any


Just type away at the command line. Make a backup of whatever you want
to make a backup of.

When you have DONE it, then put your commands, the ones you used on the
command line, in a script.

Then examine your script, and replace any particular names such as the
files and directories you chose to back up by variables defined at the
top of the script.

Then you want to think how to generate the values of those variable
automatically.

So you START by DOING.


> books? websites?  I know how to back up, using tar(and now thanks to

Then you know about all I do.

> you, i know how to back up incremental.

Oh, I don't. Tar has a mode it calls "incremental".


-g, --listed-incremental F
create/list/extract  new   GNU-format
incremental backup

but I really don't think you want it (shudder). Mind you, if you would
investigate it and see what itt does and then tell me in what way it
fails to meet the common mans idea of "incremental" I would be
grateful! Then we could send the complaint to the FSF.


> and also now thanks to you,
> again,  I know how to name the backup file automatically.

what was wrong with

`date`.tgz

?? That's a good START.

All I did was do the extra work of looking up the parameters to
give to date to make it generate a date in the format YOU specified.

But there was nothing wrong with just using "date" on its own for a
starter, and you most certainly do know that.


> I already
> knew how to ssh into a machine without a username and password(saving
> the server ssh key on the clients machine).  So what else do i need?
> The one thing i can't figure out, is how to send the email actually,  I

Send an email now, to someone.  When you have "figured" out how you did
it, add a line like that command you used to the sscript.

> normally know how to send emails when commands are successful from

Good. Then send one when successful. Be happy.

Then modify the script to send one when unsuccessful.


> command line, but i am not sure how to do it if it sftped corrected.

"corrected"?


> I
> mean, if i do it based on $? (exit status) then i may get succesful

Why not?

> everytime since the script exits the ssh server correctly.  I don't know

Maybe, maybe not! Try it and see! What's the matter? Will you be forced
to shoot your parents if the the script doesn't do it first time?
Having something which does not work is the first step on the road to
having something that does work.

> how to script it once i'm sftped in.  How to exit the ssh server via a

Who's asking you to sftp anywhere?  I'm puzzled.  If you can't do
something, don't do it!  Do something else instead!

But if you are dead set on doing something via sftp (why, oh, why?),
then you'd better read up on it. I certainly don't want to be involved
with it. What's wrong with scp or scat or plain ssh?

> script.
>
> maybe i'll have to show some "effort" and practice it right?

You'll have to TRY it! When you have difficulties, then is the time to
come back, explain the difficulties and the particular point you are
stuck on, and ask for help in getting yourself past that point.

So far, so good, no?

Peter

[ Post a follow-up to this message ]

Peter T. Breuer wrote:
> Gautam Bakshi <gautam.bakshi@gmail.com> wrote:
> 
>
>
>
> Just type away at the command line. Make a backup of whatever you want
> to make a backup of.
>
> When you have DONE it, then put your commands, the ones you used on the
> command line, in a script.
>
> Then examine your script, and replace any particular names such as the
> files and directories you chose to back up by variables defined at the
> top of the script.
>
> Then you want to think how to generate the values of those variable
> automatically.
>
> So you START by DOING.
>
>
> 
>
>
> Then you know about all I do.
>
> 
>
>
> Oh, I don't. Tar has a mode it calls "incremental".
>
>
>           -g, --listed-incremental F
>                         create/list/extract  new   GNU-format
>                         incremental backup
>
> but I really don't think you want it (shudder). Mind you, if you would
> investigate it and see what itt does and then tell me in what way it
> fails to meet the common mans idea of "incremental" I would be
> grateful! Then we could send the complaint to the FSF.
>
>
> 
>
>
> what was wrong with
>
>     `date`.tgz
>
> ?? That's a good START.
>
> All I did was do the extra work of looking up the parameters to
> give to date to make it generate a date in the format YOU specified.
>
> But there was nothing wrong with just using "date" on its own for a
> starter, and you most certainly do know that.
>
>
> 
>
>
> Send an email now, to someone.  When you have "figured" out how you did
> it, add a line like that command you used to the sscript.
>
> 
>
>
> Good. Then send one when successful. Be happy.
>
> Then modify the script to send one when unsuccessful.
>
>
> 
>
>
> "corrected"?
>
>
> 
>
>
> Why not?
>
> 
>
>
> Maybe, maybe not! Try it and see! What's the matter? Will you be forced
> to shoot your parents if the the script doesn't do it first time?
> Having something which does not work is the first step on the road to
> having something that does work.
>
> 
>
>
> Who's asking you to sftp anywhere?  I'm puzzled.  If you can't do
> something, don't do it!  Do something else instead!
>
> But if you are dead set on doing something via sftp (why, oh, why?),
> then you'd better read up on it. I certainly don't want to be involved
> with it. What's wrong with scp or scat or plain ssh?
>
> 
>
>
> You'll have to TRY it! When you have difficulties, then is the time to
> come back, explain the difficulties and the particular point you are
> stuck on, and ask for help in getting yourself past that point.
>
> So far, so good, no?
>
> Peter


yea yea yea

i think i got it to work.  I am not a programmer so let me know if there
is any problems with this script.  It seems to be doing the job.  It
seems to test fine.

I run this script, after adding the clientservers ssh key to the
backupservers trusted keys. (to do that, enter ssh-keygen -t dsa hit
enter until its done,then copy .ssh/id_dsa.pub into the backupserver's
drive then cat id_dsa.pub >> ./ssh/authorized_keys )


#!/bin/sh
filename="backup.$(date +%Y%m%d-%H%M%S).tgz"
filestobackup=`find $1 -cnewer /home/backupuser/backup.sh`
# echo $filestobackup
tar czvf $filename $filestobackup
#tar czvf \home\backupuser\$1.$(date +%Y%m%d-%H%M%S).tgz $1
touch backup.sh
#echo "file name is $filename"
scp -q $filename root@192.168.5.10:
exit $?

[ Post a follow-up to this message ]

Gautam Bakshi <gautam.bakshi@gmail.com> wrote:
> I run this script, after adding the clientservers ssh key to the
> backupservers trusted keys. (to do that, enter ssh-keygen -t dsa hit
> enter until its done,then copy .ssh/id_dsa.pub into the backupserver's
> drive then cat id_dsa.pub >> ./ssh/authorized_keys )
>
>
> #!/bin/sh
> filename="backup.$(date +%Y%m%d-%H%M%S).tgz"

You may like to do a total backup if your reference file does not
exist.

filereference="/home/backupuser/backup.time"
filestobackup="/home/backupuser/backup.list"
if [ ! -e "$filereference" ]; then
find "$1" -type f -fprint0  "$filestbackup"
else
find "$1" -cnewer "$filereference" -fprint0  "$filestbackup"
fi

> filestobackup=`find $1 -cnewer /home/backupuser/backup.sh`

That is vulnerable to filenames with spaces in (and other nasty things).
But it's OK for a start. And I suggest you list into a file instead.

But why choose to change te find line I gave, which coped with spaces in
file names by using null separators?

One probably wants to check that the list looks the right size after
finishing here.

> # echo $filestobackup
> tar czvf $filename $filestobackup

Well

tar czlpvfT --null "$filename" "$filestobackup"

or something like that.


> #tar czvf \home\backupuser\$1.$(date +%Y%m%d-%H%M%S).tgz $1
> touch backup.sh

touch "$filereference"


> #echo "file name is $filename"
> scp -q $filename root@192.168.5.10:

You probably want to do it as "backup", not "root".


> exit $?

Well, I don't generally like needless forced exits, but it won't do you
any harm!  Be aware that leaving it off would have the same result here.

Looks fine!

>
Peter

[ Post a follow-up to this message ]

In comp.os.linux.misc Peter T. Breuer <ptb@lab.it.uc3m.es>:
> Gautam Bakshi <gautam.bakshi@gmail.com> wrote: 
[vbcol=seagreen]
> You may like to do a total backup if your reference file does not
> exist.

> filereference="/home/backupuser/backup.time"
> filestobackup="/home/backupuser/backup.list"
> if [ ! -e "$filereference" ]; then
>    find "$1" -type f -fprint0  "$filestbackup"
> else
>    find "$1" -cnewer "$filereference" -fprint0  "$filestbackup"
> fi
 
[vbcol=seagreen]
> That is vulnerable to filenames with spaces in (and other nasty things).
> But it's OK for a start. And I suggest you list into a file instead.

> But why choose to change te find line I gave, which coped with spaces in
> file names by using null separators?

> One probably wants to check that the list looks the right size after
> finishing here.
 
[vbcol=seagreen]
> Well

>  tar czlpvfT --null "$filename" "$filestobackup"

> or something like that.

 
[vbcol=seagreen]
> touch "$filereference"

 
[vbcol=seagreen]
> You probably want to do it as "backup", not "root".

;)
 
[vbcol=seagreen]
> Well, I don't generally like needless forced exits, but it won't do you
> any harm!  Be aware that leaving it off would have the same result here.

if [ $? -eq "0" ]
then
echo blah | mail -s "Backup success on $(hostname)" user@remote
else
echo blah | mail -s "Backup problems on $(hostname)" user@remote
fi

> Looks fine!

Yep.

--
Michael Heiming (X-PGP-Sig > GPG-Key ID: EDD27B94)
mail: echo zvpunry@urvzvat.qr | PERL -pe 'y/a-z/n-za-m/'
#bofh excuse 309: firewall needs cooling

[ Post a follow-up to this message ]