Hi,

I try to write an application for linux. The application opens an
unknown number of connections to an unknown time and for an unknown
time. The general idea is to use non-blocking sockets to handle all
connections in one thread.

Now I got the following problem, maybe here is someone with a good idea
for a solution:

Assume there are already some connections in an fd_set and the
application is in a select-call observing this set. Now the application
makes a new connection how can I add this new connection to the set to
observe it?

The select-call waits for an IO-Event of all the other connections, but
it cannot know that there is a new connection that has to be added.

I see only three ways to solve the problem, but all this solutions are
not very nice:

- Timeout in select-call and updating fd_set after timeout.
- Using signals?.
- Using a dummy file descriptor producing an IO_Event when I need it.

I think I cannot be the only one with this problem, so maybe there is a
well known and elegant solution for it.

Thanks,
Oliver

[ Post a follow-up to this message ]

Oliver Sudmann wrote:
> Hi,
>
> I try to write an application for linux. The application opens an
> unknown number of connections to an unknown time and for an unknown
> time. The general idea is to use non-blocking sockets to handle all
> connections in one thread.
>
> Now I got the following problem, maybe here is someone with a good idea
> for a solution:
>
> Assume there are already some connections in an fd_set and the
> application is in a select-call observing this set. Now the application
> makes a new connection how can I add this new connection to the set to
> observe it?
>
> The select-call waits for an IO-Event of all the other connections, but
> it cannot know that there is a new connection that has to be added.

I'm not sure whether I understood your problem, but I try summing up
your situation:

- some sockets are open and in an fd_set
- you are currently select()ing on this fd_set

In other words, your program is blocked and - since it is
single-threaded - doing nothing.

While your program is blocked, no new connection will just fall from the
sky, so there is no problem. If your application is making a new
connection, then it is obviously not blocked on select() - and thus you
can add the new connection to the fd_set.

Greetings,
Aaron

[ Post a follow-up to this message ]

On 2006-06-08, Oliver Sudmann <nospam@nospam.com> wrote:
> I try to write an application for linux. The application opens an
> unknown number of connections to an unknown time and for an unknown
> time. The general idea is to use non-blocking sockets to handle all
> connections in one thread.

Does it mean that your application is multi-threaded?

> Assume there are already some connections in an fd_set and the
> application is in a select-call observing this set. Now the application
> makes a new connection how can I add this new connection to the set to
> observe it?

In other words, you created connection in separate thread and now want
to pass it to the thread that does select calls. I think the best in
this situation would be to use some pipe for sending signals to select.

Though even better would be to redesign your application so that no
connections are created outside of thread with select.

--
Minds, like parachutes, function best when open

[ Post a follow-up to this message ]

Hi,

> I'm not sure whether I understood your problem, but I try summing up
> your situation:
>
> - some sockets are open and in an fd_set
> - you are currently select()ing on this fd_set

yes, that is what I wanted to say.

> In other words, your program is blocked and - since it is
> single-threaded - doing nothing.
>
> While your program is blocked, no new connection will just fall from the
> sky, so there is no problem. If your application is making a new
> connection, then it is obviously not blocked on select() - and thus you
> can add the new connection to the fd_set.

No not the whole program is blocked. Only the thread with the
select-call, but there maybe other threads that create new connections.
(My fault, did not mention that there maybe other threads.) So I have to
do something to unblock the thread with the select-call.

By the way the reason why I just want to use one thread for all
connections in a multithreaded application is, because I do not want to
mess up everythings with thousends of threads for connections (imagine
e.g. a p2p-application that creates 1000 connections everyone with its
own thread)

Greetings,
Oliver

[ Post a follow-up to this message ]

Oliver Sudmann wrote:
> Hi,
> 
>
> yes, that is what I wanted to say.
> 
>
> No not the whole program is blocked. Only the thread with the
> select-call, but there maybe other threads that create new connections.
> (My fault, did not mention that there maybe other threads.) So I have to
> do something to unblock the thread with the select-call.
>
> By the way the reason why I just want to use one thread for all
> connections in a multithreaded application is, because I do not want to
> mess up everythings with thousends of threads for connections (imagine
> e.g. a p2p-application that creates 1000 connections everyone with its
> own thread)
>
> Greetings,
> Oliver

You can add your listening socket to the fd_set. When a new connection
arrives, select will end with that socket in the rd_set...in which case
you accept() the new connection and do what you want with it.

[ Post a follow-up to this message ]

Oliver Sudmann wrote:
> Hi,
> 
>
>
> yes, that is what I wanted to say.
> 
>
>
> No not the whole program is blocked. Only the thread with the
> select-call, but there maybe other threads that create new connections.
> (My fault, did not mention that there maybe other threads.) So I have to
> do something to unblock the thread with the select-call.

I think signals are the way to go. Use some data structure shared
between the threads (appropriately protected with a mutex) and a signal
as a notification telling the worker thread that "something has
happened". You can send signals to a specific thread using pthread_kill(3).

Greetings,
Aaron

[ Post a follow-up to this message ]

> I think signals are the way to go. Use some data structure shared[vbcol=seagreen]
> between the threads (appropriately protected with a mutex) and a signal
> as a notification telling the worker thread that "something has
> happened". You can send signals to a specific thread using pthread_kill(3).[/vbcol
]

To summarize your suggestions and the suggestions of Andrei Voropaev: I
should use signals for my application. But I'm really getting the
impression, that the whole thing with select is not a good solution
("even better would be to redesign your application",Andrei Voropaev).
So here is a completely different idea:

What do you think about the following alternative (completely without an
select-call) using simple polling. (Better or more worse than signals?):

Here is some pseudocode:

working thread:
while(!end)
{
for(i=0;i<number_of_connections;i++)
{
ConnectionSocket cs=list_of_connections(i);
cs.doNonBlockRecv();
cs.doNonBlockSend();
}
updateConnectionList();
Sleep();
}

A thread creating connections can add connections to another list and
everytime the for-loop has been run the connection-list of the for-loop
will be updated for the working thread.

To cut a long story short: Is polling a typical "NoNo" or can it be used
for GOOD applications?

Thanks,
Oliver

[ Post a follow-up to this message ]

Oliver Sudmann wrote: 
>
>
> To summarize your suggestions and the suggestions of Andrei Voropaev: I
> should use signals for my application. But I'm really getting the
> impression, that the whole thing with select is not a good solution
> ("even better would be to redesign your application",Andrei Voropaev).
> So here is a completely different idea:
>
> What do you think about the following alternative (completely without an
> select-call) using simple polling. (Better or more worse than signals?):
>
> Here is some pseudocode:
>
> working thread:
> while(!end)
> {
>     for(i=0;i<number_of_connections;i++)
>     {
>         ConnectionSocket cs=list_of_connections(i);
>         cs.doNonBlockRecv();
>         cs.doNonBlockSend();
>     }
>     updateConnectionList();
>     Sleep();
> }
>
> A thread creating connections can add connections to another list and
> everytime the for-loop has been run the connection-list of the for-loop
> will be updated for the working thread.
>
> To cut a long story short: Is polling a typical "NoNo" or can it be used
> for GOOD applications?
>
> Thanks,
> Oliver

This is bad design by design, IMHO.
The reason you _want_ threads is that you *want* them to be allowed to
block. If you only have one thread that deals with accept() -ing new
connections, it only needs to allocate a new "handler" thread for the
new connection (or schedule one) ,and go on with it's life: accepting
new connections.

Another reason for using threads would be using a multi processor
machine. This would only pay off if you have more than (about) four
processors: one for the app, one for the application, and two for
keeping the ethernet busy. (very course estimate, I know)

Yet another reason would be the inability to exploit poll/select.
(which can be hard, eg when recursive-descent style parsing is involved)

HTH,
AvK

[ Post a follow-up to this message ]

> In other words, you created connection in separate thread and now
> want to pass it to the thread that does select calls. I think the
> best in this situation would be to use some pipe for sending signals
> to select.

If the new connections are arriving via a listening socket, then I
would go further and advise that you NOT handle the listening socket
in a separate thread in the first place. Add the listening socket to
the set of read fds in your select call. In other words: Use threads
wisely.

b

[ Post a follow-up to this message ]

moi wrote:

> This is bad design by design, IMHO.

I could not disagree more strongly.

> The reason you _want_ threads is that you *want* them to be allowed to
> block. If you only have one thread that deals with accept() -ing new
> connections, it only needs to allocate a new "handler" thread for the
> new connection (or schedule one) ,and go on with it's life: accepting
> new connections.

How do you know the reason he wants threads? A perfectly reasonable,
common, and extremely effective use of threads is as a slight variant
to the typical single-threaded select loop design. This design is
wonderful, except it has one flaw -- if any tiny piece of code ever
blocks for any reason, your whole server collapses. Using a select loop
design with multiple threads to work around this one problem is
eminently sensible.

Of course he doesn't want his threads to block, that forces a context
switch. That's almost always less efficient than having the work be
done by the thread that's already running anyway.

DS

[ Post a follow-up to this message ]