|
Home > Archive > Unix Shell > January 2006 > Need to help make a script
You are viewing an archived Text-only version of the thread.
To view this thread in it's original format and/or if you want to reply to
this thread please [click here]
| Author |
Need to help make a script
|
|
| Herbert 2006-01-18, 8:49 pm |
| Hello,
There are 6 files. The files have time stamp like
aafile.NOV-12-09,
bbafile.NOV-12-09,
ccfile.NOV-12-09,
ddafile.NOV-12-09,
eefile.NOV-12-09,
fffile.NOV-12-09
I made a simple script to compress all the files over a month and
delete them over 6 months like below.
#!/bin/ksh
#
IMPORT_FILES= "Aafile.* \
Bbafile.* \
Ccfile.* \
Ddafile.* \
Eefile.* \
Fffile.*"
#
FILE_DIR=/xx/xx
#
#
cd $FILE_DIR
for i in $IMPORT_FILES
do
find ${i} -mtime +30 -type f -print0 | xargs -0 compress or gzip
depending on the file size -vf
done
In above script, I'd like to add a logic to check the file size and
then decide to use compress if the file size is over 2 GB MB or use
gzip if the file size is less than 2 GB. I know the recent version of
gzip can handle files oer 2 GB but it doesn't work on our system yet.
How can I make it in a single line or two lines?
I want to do the followings:
If file is older than 30 days
then
if the file size is larger than 2 GB
then
compress
else
gzip
Many thanks in advance.
Herbert
| |
| Bruce Barnett 2006-01-18, 8:49 pm |
| "Herbert" <kang_uni@hotmail.com> writes:
> There are 6 files. The files have time stamp like
> aafile.NOV-12-09,
> bbafile.NOV-12-09,
> ccfile.NOV-12-09,
> ddafile.NOV-12-09,
> eefile.NOV-12-09,
> fffile.NOV-12-09
Personally, I use a yy-mm-dd format.
It's much easier to sort, search and manipulate.
> find ${i} -mtime +30 -type f -print0 | xargs -0 compress or gzip
> depending on the file size -vf
Untested:
find ${i} -mtime +30 -type f -print0 | xargs -0 Shrink
where Shrink is
#!/bin/sh
for i in $*
do
size=`ls -l $i|awk '{print $5}'`
COMPRESS=gzip
if [ "$size" -gt 2000000 ]
then
COMPRESS=compress
fi
$COMPRESS $i
done
--
Sending unsolicited commercial e-mail to this account incurs a fee of
$500 per message, and acknowledges the legality of this contract.
| |
| Herbert 2006-01-19, 6:24 pm |
| Hello,
There are 6 files. The files have time stamp like
aafile.NOV-12-09,
bbafile.NOV-12-09,
ccfile.NOV-12-09,
ddafile.NOV-12-09,
eefile.NOV-12-09,
fffile.NOV-12-09
I made a simple script to compress all the files over a month and
delete them over 6 months like below.
#!/bin/ksh
#
IMPORT_FILES= "Aafile.* \
Bbafile.* \
Ccfile.* \
Ddafile.* \
Eefile.* \
Fffile.*"
#
FILE_DIR=/xx/xx
#
#
cd $FILE_DIR
for i in $IMPORT_FILES
do
find ${i} -mtime +30 -type f -print0 | xargs -0 compress or gzip
depending on the file size -vf
done
In above script, I'd like to add a logic to check the file size and
then decide to use compress if the file size is over 2 GB MB or use
gzip if the file size is less than 2 GB. I know the recent version of
gzip can handle files oer 2 GB but it doesn't work on our system yet.
How can I make it in a single line or two lines?
I want to do the followings:
If file is older than 30 days
then
if the file size is larger than 2 GB
then
compress
else
gzip
Many thanks in advance.
Herbert
Bruce Barnett
"Herbert" <kang_...@hotmail.com> writes:
> There are 6 files. The files have time stamp like
> aafile.NOV-12-09,
> bbafile.NOV-12-09,
> ccfile.NOV-12-09,
> ddafile.NOV-12-09,
> eefile.NOV-12-09,
> fffile.NOV-12-09
Personally, I use a yy-mm-dd format.
It's much easier to sort, search and manipulate.
> find ${i} -mtime +30 -type f -print0 | xargs -0 compress or gzip
> depending on the file size -vf
Untested:
find ${i} -mtime +30 -type f -print0 | xargs -0 Shrink
where Shrink is
#!/bin/sh
for i in $*
do
size=`ls -l $i|awk '{print $5}'`
COMPRESS=gzip
if [ "$size" -gt 2000000 ]
then
COMPRESS=compress
fi
$COMPRESS $i
done
-----------------
Bruce,
Thanks for the help
I tried it as follows:
find ${i} -mtime +90 -type f -print0 | xargs -0 rm -vf
find ${i} -mtime +14 -type f -print0 | xargs -0
# where Compression is
#!/bin/sh
for k in $*
do
SIZE="`ls -l $k | awk -F' ' '{print $5}'`"
COMP_TOOL=gzip
if [ "$SIZE" -gt 2000000000 ]
then
COMP_TOOL=compress
fi
$COMP_TOOL $k
done
done
I also tried xargs -0 Shrink. It didn't work
How can I make the output of find ( xargs -0 something) the input of
for k in $*?
Many thanks in advance!
| |
| Bruce Barnett 2006-01-19, 6:24 pm |
| "Herbert" <kang_uni@hotmail.com> writes:
> find ${i} -mtime +90 -type f -print0 | xargs -0 rm -vf
>
> find ${i} -mtime +14 -type f -print0 | xargs -0
Oops! I assume you mean
find ${i} -mtime +14 -type f -print0 | xargs -0 Compression
But it didn't work because of my mistake. I rushed.
Your example
find ${i} -mtime +14 -type f -print0 | xargs -0 Compression
won't work because files are separated by nulls and not spaces.
Two choices:
This should work
find ${i} -mtime +14 -type f | xargs Compression
But if you have filenames with spaces, new lines and semicolons nasty
things could happen. If you know this will never happen, then this
might be okay.
But at least put quotes around the filename ("$k" instead of $k), as in:
#!/bin/sh
for k in $*
do
# v--- change
SIZE="`ls -l "$k" | awk -F' ' '{print $5}'`"
COMP_TOOL=gzip
if [ "$SIZE" -gt 2000000000 ]
then
COMP_TOOL=compress
fi
$COMP_TOOL "$k" # <---change
done
done
---------------------
You can test it by first using find, and saving the output.
Then run either
Compression `cat output`
or
xargs Compression <output
(I'd change
$COMP_TOOL "$k"
to
echo $COMP_TOOL "$k"
to test this).
I guess another option is to
use the
find ...... -exec Compression '{}' \;
so that your script gets executed once per file found.
xargs can be used to improve efficiency, but it doesn't in this case.
Compression handles more than one file as an argument, but that
doesn't matter in this example.
If you really want to be efficient, use find twice
find ${i} -mtime +14 -type f -size +2000000000b | xargs compress
find ${i} -mtime +14 -type f | xargs gzip
This saves processes, and compress/gzip compresses many files in one call.
--
Sending unsolicited commercial e-mail to this account incurs a fee of
$500 per message, and acknowledges the legality of this contract.
| |
| Herbert 2006-01-19, 6:24 pm |
| "Herbert" <kang_...@hotmail.com> writes:
> find ${i} -mtime +90 -type f -print0 | xargs -0 rm -vf
> find ${i} -mtime +14 -type f -print0 | xargs -0
Oops! I assume you mean
find ${i} -mtime +14 -type f -print0 | xargs -0 Compression
But it didn't work because of my mistake. I rushed.
Your example
find ${i} -mtime +14 -type f -print0 | xargs -0 Compression
won't work because files are separated by nulls and not spaces.
Two choices:
This should work
find ${i} -mtime +14 -type f | xargs Compression
But if you have filenames with spaces, new lines and semicolons nasty
things could happen. If you know this will never happen, then this
might be okay.
But at least put quotes around the filename ("$k" instead of $k), as
in:
#!/bin/sh
for k in $*
do
# v--- change
SIZE="`ls -l "$k" | awk -F' ' '{print $5}'`"
COMP_TOOL=gzip
if [ "$SIZE" -gt 2000000000 ]
then
COMP_TOOL=compress
fi
$COMP_TOOL "$k" # <---change
done
done
---------------------
You can test it by first using find, and saving the output.
Then run either
Compression `cat output`
or
xargs Compression <output
(I'd change
$COMP_TOOL "$k"
to
echo $COMP_TOOL "$k"
to test this).
I guess another option is to
use the
find ...... -exec Compression '{}' \;
so that your script gets executed once per file found.
xargs can be used to improve efficiency, but it doesn't in this case.
Compression handles more than one file as an argument, but that
doesn't matter in this example.
If you really want to be efficient, use find twice
find ${i} -mtime +14 -type f -size +2000000000b | xargs
compress
find ${i} -mtime +14 -type f | xargs gzip
This saves processes, and compress/gzip compresses many files in one
call.
---------------------
Thanks for the reply
I tried the followings. I've got errors:
1) xarge:Compression: No such file or directory
2) where not found
find ${i} -mtime +1 -type f | xargs Compression
where Compression is
#!/bin/sh
for k in $*
do
SIZE="`ls -l "$k" | awk -F' ' '{print $5}'`"
COMP_TOOL=gzip
if [ "$SIZE" -gt 2000000000 ]
then
COMP_TOOL=compress
fi
$COMP_TOOL "$k"
done
done
One more thing:
If I do the followings:
I think there may be still a possible error at gzip if the file size is
larger than 2 GB.
find ${i} -mtime +14 -type f -size +2000000000b | xargs compress
find ${i} -mtime +14 -type f | xargs gzip
Thanks again.
| |
| Bruce Barnett 2006-01-19, 8:49 pm |
| "Herbert" <kang_uni@hotmail.com> writes:
> 1) xarge:Compression: No such file or directory
You said the name of your file was "Compression"
It's where you put it. Is it in your searchpath?
Is it excutable? (.g. you must do "chmod +x Compression")
> 2) where not found
I have no idea what this error is.
Please show the exact command and the exact output/error.
> If I do the followings:
>
> I think there may be still a possible error at gzip if the file size is
> larger than 2 GB.
>
> find ${i} -mtime +14 -type f -size +2000000000b | xargs compress
> find ${i} -mtime +14 -type f | xargs gzip
I don't understand. What is the error?
The first time, find looks for all files > 2GB, and runs compress on
them. The second time, it looks for the rest of the files. It will
skip those just compressed because the mtime is < 14 days.
--
Sending unsolicited commercial e-mail to this account incurs a fee of
$500 per message, and acknowledges the legality of this contract.
| |
| Conrad J. Sabatier 2006-01-20, 8:13 am |
| In article <1137692876.811749.78830@g49g2000cwa.googlegroups.com>,
Herbert <kang_uni@hotmail.com> wrote:
>-----------------
>Bruce,
>
>Thanks for the help
>I tried it as follows:
>
> find ${i} -mtime +90 -type f -print0 | xargs -0 rm -vf
>
> find ${i} -mtime +14 -type f -print0 | xargs -0
>
># where Compression is
>
> #!/bin/sh
>
> for k in $*
> do
> SIZE="`ls -l $k | awk -F' ' '{print $5}'`"
> COMP_TOOL=gzip
> if [ "$SIZE" -gt 2000000000 ]
> then
> COMP_TOOL=compress
> fi
> $COMP_TOOL $k
> done
>done
>
>
>I also tried xargs -0 Shrink. It didn't work
>
>How can I make the output of find ( xargs -0 something) the input of
>for k in $*?
I don't know about the Korn shell, but in bash, you can do something like
this (thereby eliminating the need for a separate script)::
find ${i} -mtime +14 -type f -print | while read
do
SIZE="`ls -l $REPLY | awk -F' ' '{print $5}'`"
COMP_TOOL=gzip
if [ "$SIZE" -gt 2000000000 ]
then
COMP_TOOL=compress
fi
$COMP_TOOL $REPLY
done
--
Conrad J. Sabatier <conrads@cox.net> -- "In Unix veritas"
| |
| Conrad J. Sabatier 2006-01-20, 8:13 am |
| In article <1137711491.108844.305210@g47g2000cwa.googlegroups.com>,
>
>I tried the followings. I've got errors:
>
>1) xarge:Compression: No such file or directory
>2) where not found
>
>
> find ${i} -mtime +1 -type f | xargs Compression
>
> where Compression is
>
> #!/bin/sh
>
> for k in $*
> do
> SIZE="`ls -l "$k" | awk -F' ' '{print $5}'`"
> COMP_TOOL=gzip
> if [ "$SIZE" -gt 2000000000 ]
> then
> COMP_TOOL=compress
> fi
>
> $COMP_TOOL "$k"
> done
>done
You're completely on the wrong track coding all of the above in a single
file.
"Compression" should be a separate script (starting with the "#!/bin/sh"
following the comment "where Compression is"). The second error you're
getting is due to the shell trying to execute the non-existent command
called "where".
--
Conrad J. Sabatier <conrads@cox.net> -- "In Unix veritas"
|
|
|
|
|