Unix Programming - Implementing read and write locks on files

Author

Implementing read and write locks on files

Roger Leigh

2005-09-25, 5:52 pm

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1

Hi,

I'm working on a program called schroot, which is used to enter chroot
environments. It also supports automatic creation and destruction of
chroot environments using LVM snapshots. A prerelease is here:
http://people.debian.org/~rleigh/schroot-0.1.6.tar.bz2

In order to support the snapshot system properly, I need to be able to
create and destroy the chroots independently of running
commands/shells inside the chroot, and I also need to keep a count of
current users so it's not possible to accidentally wipe a chroot which
is in use. This implies keeping session state somewhere.

When a chroot is created, I will write a session file to
/var/lib/schroot/sessions/$UUID.

When a chroot is entered, I will read the session specified by the
user (by UUID), increment the usage count and write it back.

When the chroot is destroyed, I will read the session specified by the
user (by UUID), and destroy it only if the usage count it 0.

Locking during each of these step is simple, if I create a separate
lockfile with O_CREAT|O_EXCL. However, there is also a command to
list the currently active sessions, which requires reading the session
files. I don't want to lock them for writing (I might block
indefinitely, since some chroot types require persistent locks), but I
do want to ensure that it's safe to read (i.e. it's not currently
being written out). How might I do this?

I think I could do this by having separate write and read locks, and
the read lock is taken by the writer to prevent reading during
modification, but not at other times. I'm not an expert by any means,
so was wondering if there were alternative approaches.

Many thanks,
Roger

- --
Roger Leigh
Printing on GNU/Linux? http://gimp-print.sourceforge.net/
Debian GNU/Linux http://www.debian.org/
GPG Public Key: 0x25BFB848. Please sign and encrypt your mail.
-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.2 (GNU/Linux)
Comment: Processed by Mailcrypt 3.5.8+ <http://mailcrypt.sourceforge.net/>

iD8DBQFDNs53VcFcaSW/ uEgRAhaYAKCCatsNiAdmpDA2zovA8PbgF5komACe
Lh45
lB4aBjqExVnucl9Ti/6bIRM=
=M6ar
-----END PGP SIGNATURE-----

Michael Rasmussen

2005-09-25, 5:53 pm

On Sun, 25 Sep 2005 17:21:13 +0100, Roger Leigh wrote:

>
> I think I could do this by having separate write and read locks, and the
> read lock is taken by the writer to prevent reading during modification,
> but not at other times. I'm not an expert by any means, so was wondering
> if there were alternative approaches.
When opening the files use open("file", O_RDONLY | O_NONBLOCK, 0). If file
is locked or empty instead of waiting forever it will return 0.

--
Hilsen/Regards
Michael Rasmussen
http://keyserver.veridis.com:11371/...arch=0xE3E80917

Kasper Dupont

2005-09-25, 5:53 pm

Michael Rasmussen wrote:
>
> On Sun, 25 Sep 2005 17:21:13 +0100, Roger Leigh wrote:
>
> When opening the files use open("file", O_RDONLY | O_NONBLOCK, 0). If file
> is locked or empty instead of waiting forever it will return 0.

Shouldn't it return -1?

--
Kasper Dupont
Note to self: Don't try to allocate
256000 pages with GFP_KERNEL on x86.

Michael Rasmussen

2005-09-25, 5:53 pm

On Sun, 25 Sep 2005 22:18:46 +0200, Kasper Dupont wrote:

>
> Shouldn't it return -1?
I was referring to the point when you start reading from the file
descriptor, so No. If returning < 0 means an error condition. When using
O_NONBLOCK it is not an error if nothing is returned which is indicated by
returning 0 - or to be precisely zero bytes. This simply means that had it
been a normal read the operation would have been blocking execution. E.G a
file was found but the file is not ready to send data jet.

--
Hilsen/Regards
Michael Rasmussen
http://keyserver.veridis.com:11371/...arch=0xE3E80917

Måns Rullgård

2005-09-25, 5:53 pm

Michael Rasmussen <mir@miras.org> writes:

> On Sun, 25 Sep 2005 22:18:46 +0200, Kasper Dupont wrote:
>
> I was referring to the point when you start reading from the file
> descriptor, so No. If returning < 0 means an error condition. When using
> O_NONBLOCK it is not an error if nothing is returned which is indicated by
> returning 0 - or to be precisely zero bytes. This simply means that had it
> been a normal read the operation would have been blocking execution. E.G a
> file was found but the file is not ready to send data jet.

It should still return -1. From the ERRORS sections of the read(2)
man page:

EAGAIN Non-blocking I/O has been selected using O_NONBLOCK and no data
was immediately available for reading.

--
Måns Rullgård
mru@inprovide.com

Kasper Dupont

2005-09-26, 2:48 am

Michael Rasmussen wrote:
>
> On Sun, 25 Sep 2005 22:18:46 +0200, Kasper Dupont wrote:
>
> I was referring to the point when you start reading from the file
> descriptor, so No. If returning < 0 means an error condition. When using
> O_NONBLOCK it is not an error if nothing is returned which is indicated by
> returning 0 - or to be precisely zero bytes. This simply means that had it
> been a normal read the operation would have been blocking execution.

Wrong. A return value of 0 from read means end of file.

--
Kasper Dupont
Note to self: Don't try to allocate
256000 pages with GFP_KERNEL on x86.

Roger Leigh

2005-09-26, 6:02 pm

Michael Rasmussen <mir@miras.org> writes:

> On Sun, 25 Sep 2005 17:21:13 +0100, Roger Leigh wrote:
>
> When opening the files use open("file", O_RDONLY | O_NONBLOCK, 0). If file
> is locked or empty instead of waiting forever it will return 0.

I'm not quite sure how this tells me the file is safe to read.
Nonblocking I/O on a block device isn't really all that meaningful, at
least as far as I understand it.

After further consideration, I think it would make more sense to do
whole-file locks using fcntl advisory locking. This supports read
(shared) and write (exclusive) locks, so I think that's a better
solution, even if it's a little more work.

This is an initial crude implementation, which appears to work OK,
though I don't like the timeout code. I'll probably replace that with
an itimer.

A signal question: if I set an itimer to deliver SIGALARM while
fcntl() is blocking on F_SETLKW, will fcntl() still be interrupted
with EINTR even if I ignore SIGALRM? (I only want to trigger it to
interrupt the call, to implement a timeout.)

static void
set_lock (int fd,
int lock_type,
guint timeout)
{
struct flock read_lock =
{
.l_type = lock_type,
.l_whence = SEEK_SET,
.l_start = 0,
.l_len = 0 // Lock entire file
};
struct timeval tv;

if (gettimeofday(&tv, NULL) < 0)
{
g_printerr(_("Failed to get current time: %s\n"), g_strerror(errno));
exit (EXIT_FAILURE);
}
tv.tv_sec += timeout; // Wait on lock for timeout seconds

while (1)
{
struct timeval tvc;

if (gettimeofday(&tvc, NULL) < 0)
{
g_printerr(_("Failed to get current time: %s\n"), g_strerror(errno));
exit (EXIT_FAILURE);
}

if (fcntl(fd, F_SETLK, &read_lock) == -1)
{
if (errno == EACCES || errno == EAGAIN)
{
if (timeval_subtract(NULL, &tv, &tvc) == TRUE)
{
g_printerr(_("Failed to acquire read lock (timeout after %u seconds): %s\n"), timeout, g_strerror(errno));
exit (EXIT_FAILURE);
}

struct timespec ts =
{
.tv_sec = 0,
.tv_nsec = 100000
};
nanosleep(&ts, NULL);
continue;
}
g_printerr(_("Failed to acquire read lock: %s\n"), g_strerror(errno));
exit (EXIT_FAILURE);
}
break;
}
}

Thanks,
Roger

--
Roger Leigh
Printing on GNU/Linux? http://gimp-print.sourceforge.net/
Debian GNU/Linux http://www.debian.org/
GPG Public Key: 0x25BFB848. Please sign and encrypt your mail.

Andrei Voropaev

2005-09-27, 7:58 am

In comp.os.linux.development.apps Roger Leigh <${rleigh}@invalid.whinlatter.ukfsn.org.invalid> wrote:
> Michael Rasmussen <mir@miras.org> writes:
>
[...]
>
> A signal question: if I set an itimer to deliver SIGALARM while
> fcntl() is blocking on F_SETLKW, will fcntl() still be interrupted
> with EINTR even if I ignore SIGALRM? (I only want to trigger it to
> interrupt the call, to implement a timeout.)

[...]

Nope. If you ignore the signal, then it never gets delivered, which
means that no system calls get interrupted. Just set as handler the
function that does nothing.

--
Minds, like parachutes, function best when open

Roger Leigh

2005-09-27, 5:55 pm

Andrei Voropaev <avorop@mail.ru> writes:

> In comp.os.linux.development.apps Roger Leigh <${rleigh}@invalid.whinlatter.ukfsn.org.invalid> wrote:
> [...]
>
> [...]
>
> Nope. If you ignore the signal, then it never gets delivered, which
> means that no system calls get interrupted. Just set as handler the
> function that does nothing.

Thanks, it works a treat!

--
Roger Leigh
Printing on GNU/Linux? http://gimp-print.sourceforge.net/
Debian GNU/Linux http://www.debian.org/
GPG Public Key: 0x25BFB848. Please sign and encrypt your mail.

Roger Leigh

2005-09-27, 5:55 pm

Roger Leigh <${rleigh}@invalid.whinlatter.ukfsn.org.invalid> writes:

> I think I could do this by having separate write and read locks, and
> the read lock is taken by the writer to prevent reading during
> modification, but not at other times. I'm not an expert by any means,
> so was wondering if there were alternative approaches.

In case anyone finds this useful, this is how I implemented whole-file
advisory locks, with lock timeouts implemented with an itimer. It
uses GLib quite extensively for error propagation.

--------sbuild-lock.h-------
/* sbuild-lock - sbuild advisory locking
*
* Copyright Â© 2005 Roger Leigh <rleigh@debian.org>
*
* schroot is free software; you can redistribute it and/or modify it
* under the terms of the GNU General Public License as published by
* the Free Software Foundation; either version 2 of the License, or
* (at your option) any later version.
*
* schroot is distributed in the hope that it will be useful, but
* WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
* General Public License for more details.
*
* You should have received a copy of the GNU General Public License
* along with this program; if not, write to the Free Software
* Foundation, Inc., 59 Temple Place, Suite 330, Boston,
* MA 02111-1307 USA
*
****************************************
*****************************/

#ifndef SBUILD_LOCK_H
#define SBUILD_LOCK_H

#include <unistd.h>
#include <fcntl.h>

#include <glib.h>

typedef enum
{
SBUILD_LOCK_ERROR_SETUP,
SBUILD_LOCK_ERROR_TIMEOUT,
SBUILD_LOCK_ERROR_FAIL
} SbuildLockError;

#define SBUILD_LOCK_ERROR sbuild_lock_error_quark()

GQuark
sbuild_auth_error_quark (void);

typedef enum
{
SBUILD_LOCK_SHARED = F_RDLCK,
SBUILD_LOCK_EXCLUSIVE = F_WRLCK,
SBUILD_LOCK_NONE = F_UNLCK
} SbuildLockType;

gboolean
sbuild_lock_set_lock (int fd,
SbuildLockType lock_type,
guint timeout,
GError **error);

gboolean
sbuild_lock_unset_lock (int fd,
GError **error);

#endif /* SBUILD_LOCK_H */

/*
* Local Variables:
* mode:C
* End:
*/
--------end of sbuild-lock.h-------

--------sbuild-lock.c-------
/* sbuild-lock - sbuild advisory locking
*
* Copyright Â© 2005 Roger Leigh <rleigh@debian.org>
*
* schroot is free software; you can redistribute it and/or modify it
* under the terms of the GNU General Public License as published by
* the Free Software Foundation; either version 2 of the License, or
* (at your option) any later version.
*
* schroot is distributed in the hope that it will be useful, but
* WITHOUT ANY WARRANTY; without even the implied warranty of
* MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU
* General Public License for more details.
*
* You should have received a copy of the GNU General Public License
* along with this program; if not, write to the Free Software
* Foundation, Inc., 59 Temple Place, Suite 330, Boston,
* MA 02111-1307 USA
*
****************************************
*****************************/

/**
* SECTION:sbuild-lock
* @short_description: advisory locking
* @title: Advisory Locking
*
* These functions implement simple whole-file shared and exclusive
* advisory locking based upon POSIX fcntl byte region locks.
*/

#include <config.h>

#define _GNU_SOURCE
#include <errno.h>
#include <stdlib.h>

#include <sys/time.h>
#include <sys/types.h>
#include <signal.h>

#include <glib.h>
#include <glib/gi18n.h>

#include "sbuild-lock.h"

/**
* sbuild_lock_error_quark:
*
* Get the SBUILD_LOCK_ERROR domain number.
*
* Returns the domain.
*/
GQuark
sbuild_lock_error_quark (void)
{
static GQuark error_quark = 0;

if (error_quark == 0)
error_quark = g_quark_from_static_string ("sbuild-lock-error-quark");

return error_quark;
}

/**
* sbuild_lock_alarm_handler:
* @ignore: the signal number.
*
* Handle the SIGALRM signal.
*/
static void
sbuild_lock_alarm_handler (int ignore)
{
/* Do nothing. This only exists so that system calls get
interrupted. */
}

/**
* sbuild_lock_clear_alarm:
* @orig_sa: the original signal handler.
*
* Restore the state of SIGALRM prior to starting lock acquisition.
*/
static void
sbuild_lock_clear_alarm (struct sigaction *orig_sa)
{
/* Restore original handler */
sigaction (SIGALRM, orig_sa, NULL);
}

/**
* sbuild_lock_set_alarm:
* @orig_sa: the original signal handler
*
* Set the SIGALARM handler. The old signal handler is stored in
* @orig_sa.
*/
static gboolean
sbuild_lock_set_alarm (struct sigaction *orig_sa)
{
struct sigaction new_sa;
sigemptyset(&new_sa.sa_mask);
new_sa.sa_flags = 0;
new_sa.sa_handler = sbuild_lock_alarm_handler;

if (sigaction(SIGALRM, &new_sa, orig_sa) != 0)
{
return FALSE;
}

return TRUE;
}

/**
* sbuild_lock_set_lock:
* @fd: the file descriptor to lock.
* @lock_type: the type of lock to set.
* @timeout: the time in seconds to wait for the lock.
* @error: a #GError.
*
* Set an advisory lock on a file. A byte region lock is placed on
* the entire file, regardless of size, using fcntl.
*
* Returns TRUE on success, FALSE on failure (@error will be set to
* indicate the cause of the failure).
*/
gboolean
sbuild_lock_set_lock (int fd,
SbuildLockType lock_type,
guint timeout,
GError **error)
{
struct flock read_lock =
{
.l_type = lock_type,
.l_whence = SEEK_SET,
.l_start = 0,
.l_len = 0 // Lock entire file
};

struct itimerval timeout_timer =
{
.it_interval =
{
.tv_sec = 0,
.tv_usec = 0
},
.it_value =
{
.tv_sec = timeout,
.tv_usec = 0
}
};

struct itimerval disable_timer =
{
.it_interval =
{
.tv_sec = 0,
.tv_usec = 0
},
.it_value =
{
.tv_sec = 0,
.tv_usec = 0
}
};

struct sigaction saved_signals;
if (sbuild_lock_set_alarm(&saved_signals) == FALSE)
{
g_set_error(error,
SBUILD_LOCK_ERROR, SBUILD_LOCK_ERROR_SETUP,
_("failed to set timeout handler: %s\n"),
g_strerror(errno));
return FALSE;
}

if (setitimer(ITIMER_REAL, &timeout_timer, NULL) == -1)
{
g_set_error(error,
SBUILD_LOCK_ERROR, SBUILD_LOCK_ERROR_SETUP,
_("failed to set timeout: %s\n"),
g_strerror(errno));
return FALSE;
}

/* Now the signal handler and itimer are set, the function can't
return without stopping the timer and restoring the signal
handler to its original state. */

/* Wait on lock until interrupted by a signal if a timeout was set,
otherwise return immediately. */
if (fcntl(fd,
(timeout != 0) ? F_SETLKW : F_SETLK,
&read_lock) == -1)
{
if (errno == EINTR)
g_set_error(error,
SBUILD_LOCK_ERROR, SBUILD_LOCK_ERROR_TIMEOUT,
_("failed to acquire lock (timeout after %u seconds)\n"), timeout);
else
g_set_error(error,
SBUILD_LOCK_ERROR, SBUILD_LOCK_ERROR_FAIL,
_("failed to acquire lock: %s\n"), g_strerror(errno));
}

if (setitimer(ITIMER_REAL, &disable_timer, NULL) == -1)
{
if (*error == NULL) /* Don't set error if already set. */
g_set_error(error,
SBUILD_LOCK_ERROR, SBUILD_LOCK_ERROR_SETUP,
_("failed to unset timeout: %s\n"),
g_strerror(errno));
}

sbuild_lock_clear_alarm(&saved_signals);

if (*error)
return FALSE;
else
return TRUE;
}

/**
* sbuild_lock_unset_lock:
* @fd: the file descriptor to unlock.
* @error: a #GError.
*
* Remove an advisory lock on a file. This is equivalent to calling
* sbuild_lock_set_lock with a lock type of SBUILD_LOCK_NONE and a
* timeout of 0.
*
* Returns TRUE on success, FALSE on failure (@error will be set to
* indicate the cause of the failure).
*/
gboolean
sbuild_lock_unset_lock (int fd,
GError **error)
{
return sbuild_lock_set_lock(fd, SBUILD_LOCK_NONE, 0, error);
}

/*
* Local Variables:
* mode:C
* End:
*/
--------end of sbuild-lock.c-------

--
Roger Leigh
Printing on GNU/Linux? http://gimp-print.sourceforge.net/
Debian GNU/Linux http://www.debian.org/
GPG Public Key: 0x25BFB848. Please sign and encrypt your mail.

JosephKK

2005-10-09, 5:52 pm

Kasper Dupont wrote:

> Michael Rasmussen wrote:
>
> Wrong. A return value of 0 from read means end of file.
>
I dug it up with info read. It will return 0 with EWOULDBLOCK or EAGAIN
while a real eof would also set FEOF.
--
JosephKK

joe@invalid.address

2005-10-09, 8:50 pm

JosephKK <joseph_barrett@sbcglobal.net> writes:

> Kasper Dupont wrote:
>
[vbcol=seagreen]
> I dug it up with info read. It will return 0 with EWOULDBLOCK or
> EAGAIN while a real eof would also set FEOF.

I think you misunderstood it. It says

"A value of zero indicates end-of-file (except if the value of the
SIZE argument is also zero). This is not considered an error. If
you keep calling `read' while at end-of-file, it will keep
returning zero and doing nothing else."

Note the "This is not considered an error" part. If it's not an error
EAGAIN won't be set.

"`EAGAIN'
Normally, when no input is immediately available, `read'
waits for some input. But if the `O_NONBLOCK' flag is set
for the file (*note File Status Flags:

, `read' returns
immediately without reading any data, and reports this
error."

Note, this is an error, so the return value would be -1, not 0.

Also, feof() is a stdio function, while read() is a system call, and
will never do anything with feof().

Joe
--
Gort, klatu barada nikto

Andrei Voropaev

2005-10-10, 7:53 am

In comp.os.linux.development.apps JosephKK <joseph_barrett@sbcglobal.net> wrote:
> Kasper Dupont wrote:
>
[...]
> I dug it up with info read. It will return 0 with EWOULDBLOCK or EAGAIN
> while a real eof would also set FEOF.

On which system? It's not true for Linux.

--
Minds, like parachutes, function best when open

JosephKK

2005-10-24, 3:48 pm

Andrei Voropaev wrote:

> In comp.os.linux.development.apps JosephKK <joseph_barrett@sbcglobal.net>
> wrote:
> [...]
>
> On which system? It's not true for Linux.
>

How very strange; the info lookup was done on a Suse 9.2 Linux system. It
also states that this is done for POSIX compliance, which many Linux
vendors are chasing for financial reasons. try info read on your Linux
system before making guesses about what it might say.

--
JosephKK