Hi,

I have found a problem with reusing UDP ports which I can't
understand.

I have a UDP server application which lets the kernel allocate an
unused port by calling bind() with a sockaddr structure setup with
my_addr.sin_port = htons(0).  This works fine and I always get an
unused port (lets say port X).  When my server application terminates
I call close() on the socket descriptor and I check the return and
find no errors.

The problem comes about when I try to reuse port X.  If another server
application binds to port X *before* any clients send datagrams to
that port, everything is fine and when clients start sending datagrams
they arrive safely.  However, if the client sends data to port X
before the new server has bound to the port then when the server comes
to bind to the port it finds that it is already in use.

I don't observe this problem when the port is allocated by hand - it
only occurs when the port is allocated by the kernel.  It seems
strange that the problem only appears when allowing the kernel to
allocate the port number and makes me think it is doing something else
behind the scenes which means that close() does not work as expected.

I'm using Linux RetHat 9, if that makes any difference.

Thanks for any suggestions.

Mark Stevens


The server code which allocates a free port is shown below:

int ListenSock;
struct sockaddr_in ServAddr;
struct sockaddr_in ResultAddr;
int ResultAddr_len;

if ((ListenSock = socket(PF_INET, SOCK_DGRAM, 0)) < 0)
{
printf("Cannot create socket errno %d\n", errno);
perror( "");
exit(1);
}

/*  bind the local address to the socket  */
memset((void *)&ServAddr, 0, sizeof(ServAddr));
ServAddr.sin_family = AF_INET;
ServAddr.sin_addr.s_addr = htonl(INADDR_ANY);
ServAddr.sin_port = htons(0);

if (bind(ListenSock, (struct sockaddr *)&ServAddr, sizeof(ServAddr))
< 0)
{
printf("Failed to bind local address to socket errno %d\n",
errno);
perror( "");
exit(1);
}

ResultAddr_len = sizeof(ResultAddr);
getsockname(ListenSock, (struct sockaddr *)&ResultAddr,
&ResultAddr_len );

printf("Kernel allocated port: %u\n", ntohs(ResultAddr.sin_port));

if (close(ListenSock) != 0)
printf("Error closing socket descriptor.\n");

[ Post a follow-up to this message ]

mas78910@hotmail.com (Mark Stevens) writes:
quote:

> I have found a problem with reusing UDP ports which I can't
> understand.
>
> I have a UDP server application which lets the kernel allocate an
> unused port by calling bind() with a sockaddr structure setup with
> my_addr.sin_port = htons(0).  This works fine and I always get an
> unused port (lets say port X).  When my server application terminates
> I call close() on the socket descriptor and I check the return and
> find no errors.
>
> The problem comes about when I try to reuse port X.  If another server
> application binds to port X *before* any clients send datagrams to
> that port, everything is fine and when clients start sending datagrams
> they arrive safely.  However, if the client sends data to port X
> before the new server has bound to the port then when the server comes
> to bind to the port it finds that it is already in use.


Try setting SO_REUSEADDR with setsockopt().
quote:

> I don't observe this problem when the port is allocated by hand - it
> only occurs when the port is allocated by the kernel.  It seems
> strange that the problem only appears when allowing the kernel to
> allocate the port number and makes me think it is doing something else
> behind the scenes which means that close() does not work as expected.
>
> I'm using Linux RetHat 9, if that makes any difference.


Using redhat can sometimes make the best of code stop working.

--
Måns Rullgård
mru@kth.se

[ Post a follow-up to this message ]

On 2 Dec 2003 02:35:55 -0800, mas78910@hotmail.com (Mark Stevens)
wrote:
quote:

>Hi,
>
>I have found a problem with reusing UDP ports which I can't
>understand.
>
>I have a UDP server application which lets the kernel allocate an
>unused port by calling bind() with a sockaddr structure setup with
>my_addr.sin_port = htons(0).  This works fine and I always get an
>unused port (lets say port X).  When my server application terminates
>I call close() on the socket descriptor and I check the return and
>find no errors.


So far, so good.
quote:

>The problem comes about when I try to reuse port X.  If another server
>application binds to port X *before* any clients send datagrams to
>that port, everything is fine and when clients start sending datagrams
>they arrive safely.  However, if the client sends data to port X
>before the new server has bound to the port then when the server comes
>to bind to the port it finds that it is already in use.


Now, with a bit of work, I can replicate the behaviour you describe
(though I'm not quite sure why things are done this way).  But what
you are doing doesn't seem to make sense in the light of your first
paragraph.  You say that you choose an ephemeral port by binding to
port 0.  That's fine.  But in the second para you talk about another
server binding to that same port.  Why are you doing this -- why does
it matter to this server that it binds to the same ephemeral port as
the earlier server invocation?  And why do you have clients sending
datagrams to a port to which there is no server?  (I ask this because
it sounds like the problem may be solved by a different -- perhaps
better -- application design.)

(By the way, SO_REUSEADDR seems to have no effect on this behaviour.)
quote:

>I don't observe this problem when the port is allocated by hand - it
>only occurs when the port is allocated by the kernel.


Yep - that's what I see too - no problem if we bind to specieid ports,
rather than using ephemeral ports.
quote:

>It seems
>strange that the problem only appears when allowing the kernel to
>allocate the port number and makes me think it is doing something else
>behind the scenes which means that close() does not work as expected.
>
>I'm using Linux RetHat 9, if that makes any difference.


I replicated your behaviour on SUSE 9.0, so it's not RH specific.

Cheers,

Michael

[ Post a follow-up to this message ]

mru@kth.se (Måns Rullgård) wrote in message news:<yw1xn0abv6wx.fsf@kth.se>...
quote:

> mas78910@hotmail.com (Mark Stevens) writes:
> 
>
> Try setting SO_REUSEADDR with setsockopt().
>


I have tried that and it makes no difference.

[ Post a follow-up to this message ]

Michael Kerrisk <michael-dot-kerrisk-at-gmx-dot-net@NOSPAM.COM> wrote in mes
sage news:<f45psv89lfd5sj2ga5omoau1hsbjg33f50@4ax.com>...

<snip>
quote:

>
> Now, with a bit of work, I can replicate the behaviour you describe
> (though I'm not quite sure why things are done this way).  But what
> you are doing doesn't seem to make sense in the light of your first
> paragraph.  You say that you choose an ephemeral port by binding to
> port 0.  That's fine.  But in the second para you talk about another
> server binding to that same port.  Why are you doing this -- why does
> it matter to this server that it binds to the same ephemeral port as
> the earlier server invocation?  And why do you have clients sending
> datagrams to a port to which there is no server?  (I ask this because
> it sounds like the problem may be solved by a different -- perhaps
> better -- application design.)
>
> (By the way, SO_REUSEADDR seems to have no effect on this behaviour.)
>


Thanks for looking into this - it gives me some consolation to know
that it's not just my code.

As to why it's being done this way:  The real situation is more
complicated than I describe in my orignal post, but there I wanted to
convey the problem without clouding the issue with details.

I have lots of client processes which need to be able to send
datagrams to a server, starting and stopping transmission at the
request of a user.  It is possible that many of these clients will be
sending data to the server at once.  If these client processes all
send their data to a single port at the server then the server needs
to untangle where these datagrams come from in order for them to be
handled properly.

To avoid this difficulty I have the following scheme:  a management
process finds a free port using bind(0), as described in my original
post.  It notes the port number and closes the matching socket
descriptor.  The management process then communicates the port number
and server name to the client that is required to start sending data,
it also communciates the port number to the server to start receiving
datagrams on.  The client then starts sending data to the specified
port on the server and the server starts 'listening' to the specified
port.  In this way the server knows that any data received on a
particular port came from a particular client and can thus deal with
it very easily.

This might sound overly complicated but the software project I'm
working on already has the infrastructure in place to make it the
easiest solution.

I'm certain that a lot of people will disagree with this design, but I
don't want to draw attention away from my main issue, which is the
inconsistency in behaviour between kernel-allocated and hand-allocated
ports.

Thanks,

Mark Stevens

[ Post a follow-up to this message ]

"Mark Stevens" <mas78910@hotmail.com> wrote in message
news:c4a36f.0312040153.44b39da1@posting.google.com...
quote:

> I have lots of client processes which need to be able to send
> datagrams to a server, starting and stopping transmission at the
> request of a user.  It is possible that many of these clients will be
> sending data to the server at once.  If these client processes all
> send their data to a single port at the server then the server needs
> to untangle where these datagrams come from in order for them to be
> handled properly.


There are two easy ways to do this. One is to use 'recvmsg', which tells
you the sending IP and port. Another is to include the sender information in
the data in each packet.
quote:

> To avoid this difficulty I have the following scheme:  a management
> process finds a free port using bind(0), as described in my original
> post.  It notes the port number and closes the matching socket
> descriptor.  The management process then communicates the port number
> and server name to the client that is required to start sending data,
> it also communciates the port number to the server to start receiving
> datagrams on.  The client then starts sending data to the specified
> port on the server and the server starts 'listening' to the specified
> port.  In this way the server knows that any data received on a
> particular port came from a particular client and can thus deal with
> it very easily.

quote:

> This might sound overly complicated but the software project I'm
> working on already has the infrastructure in place to make it the
> easiest solution.


It is overly complicated. It defeats many of the advantages of UDP.
quote:

> I'm certain that a lot of people will disagree with this design, but I
> don't want to draw attention away from my main issue, which is the
> inconsistency in behaviour between kernel-allocated and hand-allocated
> ports.


Well, your design is busted anyway. What prevents another application
(or even your same server application) from reusing the port before the
intended application can re-open it?

DS

[ Post a follow-up to this message ]

"David Schwartz" <davids@webmaster.com> wrote in message news:<bqn7am$ji0$1@
nntp.webmaster.com>...
quote:

>
>     It is overly complicated. It defeats many of the advantages of UDP.
> 
>
>     Well, your design is busted anyway. What prevents another application
> (or even your same server application) from reusing the port before the
> intended application can re-open it?
>
>     DS


I posted originally because I didn't understand something about UDP.
That is still true, regardless of the design of the project than
incorporates it.  The current state is that Micheal has taken the time
to replicated the behaviour I observe, but no one has been able to
explain it, or suggest how to fix it.  Do you have any suggestions
about this, David?

Looking forward to your reply.

Mark Stevens

[ Post a follow-up to this message ]

"Mark Stevens" <mas78910@hotmail.com> wrote in message
news:c4a36f.0312050128.200d4b36@posting.google.com...
quote:

> I posted originally because I didn't understand something about UDP.
> That is still true, regardless of the design of the project than
> incorporates it.  The current state is that Micheal has taken the time
> to replicated the behaviour I observe, but no one has been able to
> explain it, or suggest how to fix it.  Do you have any suggestions
> about this, David?
>
> Looking forward to your reply.


I don't yet understand the problem you're describing well enough to even
be sure I'd replicated the right thing if I did replicate it. That's why I'm
asking clarifying questions and pointing out other problems in your design.
Years of experience have shown me that quite often, when you fix the other
problems, the one you were worried about goes away too.

If Michael Kerris could post his code that he believes replicates your
problem, that would be very helpful. You could confirm that his code in fact
replicates the same problem and I could see if he's genuinely found a
problem.

DS

[ Post a follow-up to this message ]

In article <c4a36f.0312040153.44b39da1@posting.google.com>,
mas78910@hotmail.com (Mark Stevens) wrote:
quote:

> To avoid this difficulty I have the following scheme:  a management
> process finds a free port using bind(0), as described in my original
> post.  It notes the port number and closes the matching socket
> descriptor.  The management process then communicates the port number
> and server name to the client that is required to start sending data,
> it also communciates the port number to the server to start receiving
> datagrams on.  The client then starts sending data to the specified
> port on the server and the server starts 'listening' to the specified
> port.  In this way the server knows that any data received on a
> particular port came from a particular client and can thus deal with
> it very easily.


Does the problem happen if the manually-assigned port is bound first by
the management process, closed, and then the server tries to bind it
after the client has sent a packet?  If so, I think the problem is with
kernel- versus manually-allocated ports, but with the kernel keeping the
port associated with the original socket for some time period.
quote:

> This might sound overly complicated but the software project I'm
> working on already has the infrastructure in place to make it the
> easiest solution.


I can't really explain the behavior you're seeing (other than the above
guess) or offer a quick fix.  So perhaps some alternate design ideas
would be helpful:

1. The solution that is closest to your design is to have the management
process send the bound socket to the server using sendmsg(), rather than
just sending it the port number.  The management process can close its
descriptor after sending it to the server.

2. The server can call bind() and send the port number to the management
process, which sends it to the client.

--
Barry Margolin, barmar@alum.mit.edu
Woburn, MA

[ Post a follow-up to this message ]

Barry Margolin <barmar@alum.mit.edu> wrote in message news:<barmar-E35B25.00
084406122003@netnews.attbi.com>...
quote:

>
> Does the problem happen if the manually-assigned port is bound first by
> the management process, closed, and then the server tries to bind it
> after the client has sent a packet?


No.  This behaviour is only observed with kernel-allocated ports.

quote:

> If so, I think the problem is with
> kernel- versus manually-allocated ports, but with the kernel keeping the
> port associated with the original socket for some time period.


I think you are right about this.  The issue that I am uncertain about
is whether this is an intentional feature of UNIX sockets.  If it is,
which seems unlikely given that it is inconsistent with manual
allocation of port numbers, then there must be a way to avoid the
problem.
quote:

>
> I can't really explain the behavior you're seeing (other than the above
> guess) or offer a quick fix.  So perhaps some alternate design ideas
> would be helpful:
>
> 1. The solution that is closest to your design is to have the management
> process send the bound socket to the server using sendmsg(), rather than
> just sending it the port number.  The management process can close its
> descriptor after sending it to the server.
>
> 2. The server can call bind() and send the port number to the management
> process, which sends it to the client.


If I can't find a solution to the problem then I will have to think
about doing something like you suggest.  Thanks for your ideas.


Mark Stevens

[ Post a follow-up to this message ]