how to make a parent and all its children write to a file without race
condition and data inconsistency

[ Post a follow-up to this message ]

googler wrote:
> how to make a parent and all its children write to a file without race
> condition and data inconsistency
>

man flock, man lockf, man fcntl

--
Lev Walkin
vlm@lionet.info

[ Post a follow-up to this message ]

On 14 Jul 2004 22:32:50 -0700 "googler" <arun_ccjl@yahoo.co.in> wrote:
> how to make a parent and all its children write to a file without race
> condition and data inconsistency

choose /dev/null as the file

[ Post a follow-up to this message ]

>how to make a parent and all its children write to a file without race
>condition and data inconsistency

On at least some UNIX systems (Solaris),
lseek(...,SEEK_END)
followed by
write(...)

writes the the end of the file at the time of the write, not at the time
of the lseek. Several such writes will not clobber each other.
--
mac the naïf

[ Post a follow-up to this message ]

"googler" <arun_ccjl@yahoo.co.in> wrote in message news:cd54u2$scd@odbk17.pr
od.google.com...
: how to make a parent and all its children write to a file without race
: condition and data inconsistency
:

Ignore the other answers.  Open the file with the O_APPEND flag:

from man write(2):

If the O_APPEND flag of the file status flags  is  set,  the
file offset will be set to the end of the file prior to each
write and no intervening file  modification  operation  will
occur  between changing the file offset and the write opera-
tion.


Dan Mercer

[ Post a follow-up to this message ]

Dan Mercer <dmercer@mn.rr.com> wrote:

> "googler" <arun_ccjl@yahoo.co.in> wrote in message news:cd54u2$scd@odbk17.
prod.google.com...
> : how to make a parent and all its children write to a file without race
> : condition and data inconsistency
> :

> Ignore the other answers.  Open the file with the O_APPEND flag:

> from man write(2):

or open(2) (at least on my machine)

>      If the O_APPEND flag of the file status flags  is  set,  the
>      file offset will be set to the end of the file prior to each
>      write and no intervening file  modification  operation  will
>      occur  between changing the file offset and the write opera-
>      tion.

Some man pages warn that this may not work with NFS-mounted files
since at least some NFS implementations do not support appending
and the client kernel has to fake it by lseek/write calls which
are then open to the race condition again. Then a lock with fcntl()
might be the only save thing (given that the NFS implementation
supports fcntl()-locks).
Regards, Jens
--
\   Jens Thoms Toerring  ___  Jens.Toerring@physik.fu-berlin.de
\__________________________  http://www.toerring.de

[ Post a follow-up to this message ]

Dan Mercer wrote:
> "googler" <arun_ccjl@yahoo.co.in> wrote in message news:cd54u2$scd@odbk17.
prod.google.com...
> : how to make a parent and all its children write to a file without race
> : condition and data inconsistency
> :
>
> Ignore the other answers.  Open the file with the O_APPEND flag:
>
> from man write(2):
>
>      If the O_APPEND flag of the file status flags  is  set,  the
>      file offset will be set to the end of the file prior to each
>      write and no intervening file  modification  operation  will
>      occur  between changing the file offset and the write opera-
>      tion.

But note that this leaves room (as I read it) for multiple processes to
compete for non-atomic writes. I.e. if one process opens the file for
append and then makes two write() system calls, each is guaranteed
atomic but the two together are not; another process could manage to
insert some data between them. This is particularly a concern when using
stdio since buffering is imposed under the covers and thus what looks
like a single fwrite/fprintf to the user may in fact result in multiple
write() system calls.

In my app I use a combination of O_APPEND and fcntl locking and it seems
to work fine (but my app only writes to local files).

--
Thanks,
M.Biswas

[ Post a follow-up to this message ]

On Fri, 16 Jul 2004 15:48:25 +0000, Alex Colvin wrote:
 
>
> On at least some UNIX systems (Solaris),
> 	lseek(...,SEEK_END)
> followed by
> 	write(...)
>
> writes the the end of the file at the time of the write, not at the time
> of the lseek.

This is true.

>               Several such writes will not clobber each other.

This is not true.
The "end of file" will be relative to the end of the file at the time
lseek was called. So multiple processes might run the lseek call, then
each will start the write at the same position.

--
James Antill -- james@and.org
Need an efficient and powerful string library for C?
http://www.and.org/vstr/

[ Post a follow-up to this message ]

On Fri, 16 Jul 2004 17:02:17 -0400, James Antill
<james-netnews@and.org> wrote:

>On Fri, 16 Jul 2004 15:48:25 +0000, Alex Colvin wrote:
> 
>
> This is true.
> 
>
> This is not true.
> The "end of file" will be relative to the end of the file at the time
>lseek was called.

No - as above, the end of file is that at the time write() is called.

>So multiple processes might run the lseek call, then
>each will start the write at the same position.

No, each of them will atomically seek to the current EOF, append their
data, then the next one will write at the new EOF. They won't
overwrite each other unless they're running over NFS (which introduces
a race condition) or they used lseek with an explicit offset rather
than SEEK_END.


James.

[ Post a follow-up to this message ]

On Sat, 17 Jul 2004 16:34:32 +0100, James wrote:

> On Fri, 16 Jul 2004 17:02:17 -0400, James Antill
> <james-netnews@and.org> wrote:
> 
>
> No - as above, the end of file is that at the time write() is called.

No, it writes the end of the file ... so if you extend the file, the hole
isn't created by the seek. But the return value is defined as:

Upon successful completion, the resulting offset,  as  meas-
ured  in  bytes from the beginning of the file, is returned.

...this means the seek to END can't happen in an atomic transaction with
the write call. O_APPEND is there for a reason.
 
>
> No, each of them will atomically seek to the current EOF, append their
> data, then the next one will write at the new EOF.

While I don't currently have access to try it myself on a Solaris
box, feel free to run a few instances of the following. I'm 99% sure
Solaris isn't completely broken in the way you suggest, and the file will
only grow by 50 bytes for the entire interval...

#include <sys/types.h>
#include <sys/stat.h>
#include <fcntl.h>
#include <unistd.h>
#include <stdlib.h>
#include <err.h>

#define CONF_FNAME "abcd"
#define CONF_DATA "123456789 123456789 123456789 123456789 1234567890"
#define CONF_INTERVAL 10

int main(void)
{
int fd = open(CONF_FNAME, O_WRONLY);

if (fd == -1) err(EXIT_FAILURE, "open");
if (lseek(fd, 0, SEEK_END) == -1)
err(EXIT_FAILURE, "seek");
sleep(CONF_INTERVAL);
if (write(fd, CONF_DATA, sizeof(CONF_DATA) - 1) == -1)
err(EXIT_FAILURE, "write");

return EXIT_SUCCESS;
}

--
James Antill -- james@and.org
Need an efficient and powerful string library for C?
http://www.and.org/vstr/

[ Post a follow-up to this message ]