|
Home > Archive > Unix Shell > January 2006 > Autonomy for cp and mv
You are viewing an archived Text-only version of the thread.
To view this thread in it's original format and/or if you want to reply to
this thread please [click here]
| Author |
Autonomy for cp and mv
|
|
|
| When copying or moving a file I expect it is doing a fread from source
and fwrite to destination.
Other than writing a program that puts a semaphore in place and
checking for it, is there something I can do in the shell that can
guarantee me that my file is complete (moving or copying) before
another process comes along and reads it.
I don't want the second process to get an incomplete file.
| |
| Benjamin Schieder 2006-01-29, 9:30 pm |
| Stu wrote:
> When copying or moving a file I expect it is doing a fread from source
> and fwrite to destination.
>
> Other than writing a program that puts a semaphore in place and
> checking for it, is there something I can do in the shell that can
> guarantee me that my file is complete (moving or copying) before
> another process comes along and reads it.
>
> I don't want the second process to get an incomplete file.
It doesn't. As long as there's an open filedescriptor on the file, it won't be
deleted. This means you can do stuff like this:
mplayer /some/file.avi &
rm -f /some/file.avi
cat /proc/`pidof mplayer`/fd/${some file descriptor} > /some/file.avi
and the file will be there again.
Greetings,
Benjamin
--
Today, memory either forgets things when you don't want it to,
or remembers things long after they're better forgotten.
| |
| Robert Bonomi 2006-01-29, 9:30 pm |
| In article <1138220096.069204.36090@g49g2000cwa.googlegroups.com>,
Stu <beefstu350@hotmail.com> wrote:
>When copying or moving a file I expect it is doing a fread from source
>and fwrite to destination.
>
>Other than writing a program that puts a semaphore in place and
>checking for it, is there something I can do in the shell that can
>guarantee me that my file is complete (moving or copying) before
>another process comes along and reads it.
>
>I don't want the second process to get an incomplete file.
>
*As*asked*, the answer to your question is 'no'. you can't prevent
access to the destination file name while the copy is in progress.
However, what you _can_ do, is make the destination of the copy/move
a temporary file name in the target directory, and then _rename_ that
temp-file to the real name after the copy/move is complete.
| |
| Chris F.A. Johnson 2006-01-29, 9:30 pm |
| On 2006-01-25, Stu wrote:
> When copying or moving a file I expect it is doing a fread from source
> and fwrite to destination.
>
> Other than writing a program that puts a semaphore in place and
> checking for it, is there something I can do in the shell that can
> guarantee me that my file is complete (moving or copying) before
> another process comes along and reads it.
>
> I don't want the second process to get an incomplete file.
If mv is on the same file system, it is just a rename, so there's
no problem.
If you are using cp, or mv to a different filesystem, use a
different destination name, then rename with mv:
tempfile="$DEST/${FILE##*/}-tmp$$"
cp "$FILE" "$tempfile"
mv "$tempfile" "$DEST/${FILE##*/}"
--
Chris F.A. Johnson, author | <http://cfaj.freeshell.org>
Shell Scripting Recipes: | My code in this post, if any,
A Problem-Solution Approach | is released under the
2005, Apress | GNU General Public Licence
| |
| Hrvoje Spoljar 2006-01-29, 9:31 pm |
| Benjamin Schieder <blindcoder@scavenger.homeip.net> wrote at comp.unix.shell:
> It doesn't. As long as there's an open filedescriptor on the file, it won't be
> deleted. This means you can do stuff like this:
>
> mplayer /some/file.avi &
> rm -f /some/file.avi
> cat /proc/`pidof mplayer`/fd/${some file descriptor} > /some/file.avi
>
> and the file will be there again.
depending on which cat you use, on some distributions cat is not binary safe.
But you could cp that file descriptor and you would get your file.
--
____ __ ___| | ___ Ignorance is .~. hrvoje.spoljar@><.pbf.hr
(_-< '_ \/ _ \ |_/ -_) bliss, but / V \ irc # RoCkY
/__/ .__/\___/__/\___| knowledge is /( )\ icq : 53000945
|_| power! ^-^ http://spole.pbf.hr
| |
| Casper H.S. Dik 2006-01-29, 9:31 pm |
| Hrvoje Spoljar <fcbyr@k.cos.ue> writes:
>Benjamin Schieder <blindcoder@scavenger.homeip.net> wrote at comp.unix.shell:
[vbcol=seagreen]
>depending on which cat you use, on some distributions cat is not binary safe.
>But you could cp that file descriptor and you would get your file.
If cat is not binary safe it is broken. Could you name even one
such disstribution?
Casper
--
Expressed in this posting are my opinions. They are in no way related
to opinions held by my employer, Sun Microsystems.
Statements on Sun products included here are not gospel and may
be fiction rather than truth.
| |
| Stephane Chazelas 2006-01-29, 9:31 pm |
| On Thu, 26 Jan 2006 11:35:14 +0000 (UTC), Hrvoje Spoljar wrote:
> Benjamin Schieder <blindcoder@scavenger.homeip.net> wrote at comp.unix.shell:
>
> depending on which cat you use, on some distributions cat is not binary safe.
[...]
??? Where is that? I've never come accross any. POSIX requires
it to be binary safe, and I can't see how a UNIX implementation
of cat could fail to be, it's just doing a while(read) write;.
Are you thinking of some port of the tool to Apple or Microsoft
systems?
--
Stephane
| |
| Bruce Barnett 2006-01-29, 9:31 pm |
| Casper H.S. Dik <Casper.Dik@Sun.COM> writes:
> If cat is not binary safe it is broken. Could you name even one
> such disstribution?
Could someone explain this a little more? The only thing I can think
of is a sparse file. I've never tested this, but I might guess copying
a sparse file would create a new file that allocates more disk
blocks. But the two files should be identical at the binary level, right?
--
Sending unsolicited commercial e-mail to this account incurs a fee of
$500 per message, and acknowledges the legality of this contract.
| |
| Casper H.S. Dik 2006-01-29, 9:31 pm |
| Bruce Barnett <spamhater113+U060126095415@grymoire.com> writes:
>Casper H.S. Dik <Casper.Dik@Sun.COM> writes:
[vbcol=seagreen]
>Could someone explain this a little more? The only thing I can think
>of is a sparse file. I've never tested this, but I might guess copying
>a sparse file would create a new file that allocates more disk
>blocks. But the two files should be identical at the binary level, right?
"Holes in files" are an implementation artefact and they are not
detectable by portable programs. So, indeed, "cmp" will show the
files as being the same.
In case you think that you can derive "holiness" from the allocated
block count, then look at:
ls -ls zeros xes random
129 -rw------- 1 casper ir 65536 Jan 26 16:12 random
6 -rw------- 1 casper ir 65536 Jan 26 16:11 xes
1 -rw------- 1 casper ir 65536 Jan 26 16:11 zeros
This is how I created the files:
117 16:11 dd if=/dev/zero of=zeros count=128
119 16:11 PERL -e 'print "x" x 65536' > xes
121 16:12 dd if=/dev/random of=random count=128
Casper
| |
| Bruce Barnett 2006-01-29, 9:31 pm |
| Casper H.S. Dik <Casper.Dik@Sun.COM> writes:
> "Holes in files" are an implementation artefact and they are not
> detectable by portable programs. So, indeed, "cmp" will show the
> files as being the same.
>
> In case you think that you can derive "holiness" from the allocated
> block count, then look at:
>
> ls -ls zeros xes random
> 129 -rw------- 1 casper ir 65536 Jan 26 16:12 random
> 6 -rw------- 1 casper ir 65536 Jan 26 16:11 xes
> 1 -rw------- 1 casper ir 65536 Jan 26 16:11 zeros
>
> This is how I created the files:
>
> 117 16:11 dd if=/dev/zero of=zeros count=128
> 119 16:11 PERL -e 'print "x" x 65536' > xes
> 121 16:12 dd if=/dev/random of=random count=128
Thanks, Casper. I tried the same on my Linux 2.6.12 system,
(using dd if=/dev/urandom of=urandom count=128)
and got
64 -rw------- 1 barnett grp1 65536 Jan 26 14:17 urandom
64 -rw------- 1 barnett grp1 65536 Jan 26 13:59 xes
64 -rw------- 1 barnett grp1 65536 Jan 26 13:59 zeros
--
Sending unsolicited commercial e-mail to this account incurs a fee of
$500 per message, and acknowledges the legality of this contract.
| |
| Barry Margolin 2006-01-29, 9:31 pm |
| In article <draobn$tfc$2$208.20.133.66@netheaven.com>,
Bruce Barnett <spamhater113+U060126095415@grymoire.com> wrote:
> Casper H.S. Dik <Casper.Dik@Sun.COM> writes:
>
>
> Could someone explain this a little more? The only thing I can think
> of is a sparse file. I've never tested this, but I might guess copying
> a sparse file would create a new file that allocates more disk
> blocks. But the two files should be identical at the binary level, right?
Right. And if this is what the guy was talking about, the same is
probably true of some implementations of cp.
--
Barry Margolin, barmar@alum.mit.edu
Arlington, MA
*** PLEASE post questions in newsgroups, not directly to me ***
*** PLEASE don't copy me on replies, I'll read them in the group ***
|
|
|
|
|