I have a multithreaded program running on Solaris 10 U1.  I've only
tested on x86 so far.  It listens on a unix domain socket for client
requests, and returns responses on the same socket.  The client can
reuse the socket for multiple transactions.

The server has an accept thread which just does an accept() in a loop
and passes the resulting fd off to a work thread.

I have a client that connects and does one transaction then quits.
The server sees this in the work thread (read returns 0) and closes
the fd and returns (thread exits).

When this happens, and the client connects again, instead of getting
the same fd, it gets the next one.  But then on the next (third) connect,
it again gets the lowest fd.  This continues to cycle this way.

Say the listen socket is fd 4.  The client connects, in the server
this is fd 5.  The client disconnects, *the server does a close()* and
the thread exits.  pfiles shows that 4 is the highest open fd.  When
the client connects again, it is fd *6* in the server.  After disconnect,
again we're left with 4 as the highest open fd.  Then for the next
connect, the next fd is correctly 5.

It seems that the next available fd is chosen on entry to accept(),
rather than on exit.

T1                            T2
accept(4)                  does not exist
blocks, next avail 5
=5
pthread_create
accept(4)                 ... working ...
blocks, next avail 6
.. close(5), return() ...
does not exist
=6

Note: a new connection does not occur until after fd 5 is closed.
It's not a race or observation timing issue.  Under the same test,
FC3 Linux (kernel 2.6.12-1.1380_FC3 and glibc-0.2.3.6-0.fc3.1) always
returns fd 5.

Is the Solaris behavior valid SUS/POSIX?

-frank

[ Post a follow-up to this message ]

Frank Cusack schrieb:

> I have a multithreaded program running on Solaris 10 U1.  I've only
> tested on x86 so far.  It listens on a unix domain socket for client
> requests, and returns responses on the same socket.  The client can
> reuse the socket for multiple transactions.
>
> The server has an accept thread which just does an accept() in a loop
> and passes the resulting fd off to a work thread.
>
> I have a client that connects and does one transaction then quits.
> The server sees this in the work thread (read returns 0) and closes
> the fd and returns (thread exits).
>
> When this happens, and the client connects again, instead of getting
> the same fd, it gets the next one.  But then on the next (third) connect,
> it again gets the lowest fd.  This continues to cycle this way.
>
> Say the listen socket is fd 4.  The client connects, in the server
> this is fd 5.  The client disconnects, *the server does a close()* and
> the thread exits.  pfiles shows that 4 is the highest open fd.  When
> the client connects again, it is fd *6* in the server.  After disconnect,
> again we're left with 4 as the highest open fd.  Then for the next
> connect, the next fd is correctly 5.
>
> It seems that the next available fd is chosen on entry to accept(),
> rather than on exit.
>
>     T1                            T2
>     accept(4)                  does not exist
>       blocks, next avail 5
>       =5
>     pthread_create
>     accept(4)                 ... working ...
>       blocks, next avail 6
>                               ... close(5), return() ...
>                                does not exist
>       =6
>
> Note: a new connection does not occur until after fd 5 is closed.
> It's not a race or observation timing issue.  Under the same test,
> FC3 Linux (kernel 2.6.12-1.1380_FC3 and glibc-0.2.3.6-0.fc3.1) always
> returns fd 5.
>
> Is the Solaris behavior valid SUS/POSIX?
>
> -frank

You do not mean open(2), you mean accept(2). Open is required to return
the lowest available file descriptor:

http://www.opengroup.org/onlinepubs...tions/open.html

Accept has no such requirement:

http://www.opengroup.org/onlinepubs...ons/accept.html

Programs using open and dup sometimes depend on this behaviour,
especially if you close stdin/out or err and reopen them or dup
descriptors to 0, 1, 2. Returning random fds would break such programs.
Also, old unix version did not have a dup2(2) (only dup(2)) system
call, so relying on this behaviour was the only way to reassign the
standard file descriptors.

Other calls like socket(2) and accept(2) are not used in such ways.

Hubble.

[ Post a follow-up to this message ]

Frank Cusack wrote:

> It seems that the next available fd is chosen on entry to accept(),
> rather than on exit.

It is very reasonable to assume that a blocking 'accept' will almost
certainly suceed and so allocate the socket, then block until you can
'fill it in'.

> Is the Solaris behavior valid SUS/POSIX?

Yes. There is no requirement to the contrary.

The 'open' function is required to return the lowest available fd, but
that has to mean when at some arbitrary point when the call is made and
cannot mean when it returns. It would be interesting to do some tests
with blocking 'open's of RS232 ports.

Would you argue that a blocking 'open' of an RS232 port must block
before they assign a file descriptor? That seems unreasonable to me.

DS

[ Post a follow-up to this message ]

Frank Cusack <fcusack@fcusack.com> writes:

>It seems that the next available fd is chosen on entry to accept(),
>rather than on exit.

That is correct as you can verify from the source.  This is done so
accept() can return EMFILE early.

>    T1                            T2
>    accept(4)                  does not exist
>      blocks, next avail 5
>      =5
>    pthread_create
>    accept(4)                 ... working ...
>      blocks, next avail 6
>                              ... close(5), return() ...
>                               does not exist
>      =6

>Note: a new connection does not occur until after fd 5 is closed.
>It's not a race or observation timing issue.  Under the same test,
>FC3 Linux (kernel 2.6.12-1.1380_FC3 and glibc-0.2.3.6-0.fc3.1) always
>returns fd 5.

>Is the Solaris behavior valid SUS/POSIX?

Well, considering that the SUS open() manual page says:

Upon successful completion, the function will open the file and
return a non-negative integer representing the lowest numbered
unused file descriptor. Otherwise, -1 is returned and errno is
set to indicate the error. No files will be created or modified
if the function returns -1.

but the accept manual page says:

Upon successful completion, accept() returns the nonnegative
file descriptor of the accepted socket. Otherwise, -1 is
returned and errno is set to indicate the error.

one could assume it is legal.

But it does mean that Solaris allocates the file descriptor way to early,
and can return EMFILE even when no connection is pending (but even that
is perfectly vaid behaviour)

Casper
--
Expressed in this posting are my opinions.  They are in no way related
to opinions held by my employer, Sun Microsystems.
Statements on Sun products included here are not gospel and may
be fiction rather than truth.

[ Post a follow-up to this message ]

davids@webmaster.com writes:


>The 'open' function is required to return the lowest available fd, but
>that has to mean when at some arbitrary point when the call is made and
>cannot mean when it returns. It would be interesting to do some tests
>with blocking 'open's of RS232 ports.

And there's of course some interesting discussion possible as to
what "in use" means in a multi-threaded application.

If multiple concurrent system calls each of which can return a file
descriptor, will open still return the lowest numbered fd?  It
seems impossible to guarantee (two concurrent opens may return fd
3 and 4, but once the decision is made which open gets which fd, there's
no guarantee which of the two opens returns first.

>Would you argue that a blocking 'open' of an RS232 port must block
>before they assign a file descriptor? That seems unreasonable to me.

Quite.

Casper
--
Expressed in this posting are my opinions.  They are in no way related
to opinions held by my employer, Sun Microsystems.
Statements on Sun products included here are not gospel and may
be fiction rather than truth.

[ Post a follow-up to this message ]

In article <449a7d80$0$31654$e4fe514c@news.xs4all.nl>,
Casper H.S. Dik <Casper.Dik@Sun.COM> wrote:

> davids@webmaster.com writes:
>
> 
>
> And there's of course some interesting discussion possible as to
> what "in use" means in a multi-threaded application.

I think that any application that cares about this would have to do
enough mutual exclusion to make it impossible for two threads to be
calling open() concurrently.  I think the only reason any application
would care about this behavior is one that's avoiding use of dup2() to
open stdin, stdout, or stderr to a specific target.  If you did this
without mutual exclusion, you wouldn't have any way of knowing which
thread would win.  I suppose it's remotely possible that you don't care
which, you just want the specified FD to be opened by one of them.

--
Barry Margolin, barmar@alum.mit.edu
Arlington, MA
*** PLEASE post questions in newsgroups, not directly to me ***
*** PLEASE don't copy me on replies, I'll read them in the group ***

[ Post a follow-up to this message ]

Casper H.S. Dik wrote:

> Well, considering that the SUS open() manual page says:
>
> 	Upon successful completion, the function will open the file and
> 	return a non-negative integer representing the lowest numbered
> 	unused file descriptor. Otherwise, -1 is returned and errno is
> 	set to indicate the error. No files will be created or modified
> 	if the function returns -1.

Arguably, the use of the words "upon successful completion" mean that
the file descriptor must be the lowest one as of the time the open is
determined to be able to succeed. This would mean that it would,
technically, be an error for a blocking open of an RS232 port to assign
the fd first. At that time, we have no idea if the open will succeed,
and at the time that determination is made, there could be a lower fd
available.

I'll bet dollars to donuts most existing implementations violate this
rule. (Assuming it really is a rule. Who knows.)

DS

[ Post a follow-up to this message ]

>> Well, considering that the SUS open() manual page says: 
>
>Arguably, the use of the words "upon successful completion" mean that
>the file descriptor must be the lowest one as of the time the open is
>determined to be able to succeed. This would mean that it would,
>technically, be an error for a blocking open of an RS232 port to assign
>the fd first.

Ok, can an open() fail after it gets to the point of waiting for
carrier detect?  It could get interrupted with a signal (I suppose
that counts as failure).  Is it considered failure if it waits
forever for something that may never happen, depending on external
events?  Or is that just "not finished yet"?

>At that time, we have no idea if the open will succeed,
>and at the time that determination is made, there could be a lower fd
>available.
>
>I'll bet dollars to donuts most existing implementations violate this
>rule. (Assuming it really is a rule. Who knows.)
>
>DS

Gordon L. Burditt

[ Post a follow-up to this message ]

In article <1151102965.739706.272160@c74g2000cwc.googlegroups.com>,
davids@webmaster.com wrote:

> Casper H.S. Dik wrote:
> 
>
> Arguably, the use of the words "upon successful completion" mean that
> the file descriptor must be the lowest one as of the time the open is
> determined to be able to succeed. This would mean that it would,
> technically, be an error for a blocking open of an RS232 port to assign
> the fd first. At that time, we have no idea if the open will succeed,
> and at the time that determination is made, there could be a lower fd
> available.
>
> I'll bet dollars to donuts most existing implementations violate this
> rule. (Assuming it really is a rule. Who knows.)

I don't think that "upon successful completion" is intended to imply a
particular time frame.  It just means "if the call succeeds".

--
Barry Margolin, barmar@alum.mit.edu
Arlington, MA
*** PLEASE post questions in newsgroups, not directly to me ***
*** PLEASE don't copy me on replies, I'll read them in the group ***

[ Post a follow-up to this message ]

davids@webmaster.com writes:


>Casper H.S. Dik wrote:
 
[vbcol=seagreen]
>Arguably, the use of the words "upon successful completion" mean that
>the file descriptor must be the lowest one as of the time the open is
>determined to be able to succeed. This would mean that it would,
>technically, be an error for a blocking open of an RS232 port to assign
>the fd first. At that time, we have no idea if the open will succeed,
>and at the time that determination is made, there could be a lower fd
>available.

>I'll bet dollars to donuts most existing implementations violate this
>rule. (Assuming it really is a rule. Who knows.)

I don't think it's a rule.  The full phrase is upon successful completion
the funciton will "open the file and return ...".

Surely your argument implies that open() is not allowed to open a
file *until* it completes successfully? That seems rather contradictory.

It would also be bad for other reasons: opening devices and files
can have side-effects; not allocating the fd prior to opening the
file can cause an EMFILE return because we're not out of fds which
you cannot guard against other than by allocating an fd first.  Such
sideeffects could include:

- raising carrier on a modem
- creating a file (O_CREAT)
- rewinding a tape

None of these should happen when open returns -1.

Of course, an application could try:


fd' = fcntl(fd, F_DUPFD, 0);
if (fd' < fd && fd' >= 0) {
close(fd);
fd = fd';
}

(Except you'd need to use an internal form which doesn't actuall calls
close on the fd)

Pointless for something which a POSIX compliant application is
unlikely to be able to detect.

Casper
--
Expressed in this posting are my opinions.  They are in no way related
to opinions held by my employer, Sun Microsystems.
Statements on Sun products included here are not gospel and may
be fiction rather than truth.

[ Post a follow-up to this message ]