Typical problem where a while(1) consumes gobs of CPU.  Not as
straightforward (to me) to figure out how to replace it with
something more CPU friendly.

The overall system has three daemons:

1) suck_packets_d
2)  first_order_packet_filter_and_packet_buf
fer_d
3)  extract_interesting_packet_features_and_
generate_report_d

Using some sort of blocking calls in 1) or 2) seems like a bad idea,
because packet volume can be high and I want to keep up with it as
much as possible.

I could use some help thinking this through if anyone is so inclined.

C / Linux

pseudocode for daemon (2) (biggest CPU consumer)

main()
{
while(1)
{
retcode = function() ;
switch retcode
{
case	errortype1
case	errortype2
case	errortype3
}
}

return 0 ;
}

int function()
{
value = 0 ;

// value is pass-by-result here
read_incoming_svs_v_message_queue( value ) ;

if keep_new_data( value )
return write_outgoing_sys_v_message_queue() ;
else
return	NO_OP ;
}

int read_incoming_svs_v_message_queue( value )
{
if msgrcv() < 0
if errno != ENOMSG
return ERROR
else
return	NO_OP
else
{
read_inbound_posix_shared_memory() ;

while (1)
{
if fcntl() GET LOCK < 0
{
if errno == EAGAIN or errno == EACCES
continue ;
else
return	ERROR
}
}

value = read locked resource

if fcntl() UNLOCK < 0
handle_error()
}
}

int write_outgoing_sys_v_message_queue()
{
while (1)
{
if fcntl() GET LOCK < 0
{
if errno == EAGAIN or errno == EACCES
continue ;
else
return	ERROR
}
}

write to locked resource

if fcntl() UNLOCK < 0
handle_error()

if msgsnd() fails
return	ERROR ;

write to posix shared memory
}

[ Post a follow-up to this message ]

"Vendor Neutral" <neutral@domain.invalid> wrote in message
news:MPG.1c2a4f80ebc7ecbf989f9f@news.verizon.net...
> Typical problem where a while(1) consumes gobs of CPU.  Not as
> straightforward (to me) to figure out how to replace it with
> something more CPU friendly.
>
> The overall system has three daemons:
>
> 1) suck_packets_d
> 2)  first_order_packet_filter_and_packet_buf
fer_d
> 3)  extract_interesting_packet_features_and_
generate_report_d
>
> Using some sort of blocking calls in 1) or 2) seems like a bad idea,
> because packet volume can be high and I want to keep up with it as
> much as possible.

Sorry, but "blocking calls" are designed for this purpose.  Why is it a bad
idea?  If the data is there to process then you are unblocked.  These types
of semaphores are generally designed to work with the I/O and CPU to create
the best possible scenario.

[ Post a follow-up to this message ]

On Wed, 15 Dec 2004, Vendor Neutral wrote:

> Using some sort of blocking calls in 1) or 2) seems like a bad idea,

Why do you think that?

> because packet volume can be high and I want to keep up with it as
> much as possible.

Umm, if there is data available, blocking functions won't block: they'll
return data immediately.

Using functions that block if no data is available is the correct
thing to do.

--
Rich Teer, SCNA, SCSA, author of "Solaris Systems Programming"

President,
Rite Online Inc.

Voice: +1 (250) 979-1638
URL: http://www.rite-group.com/rich

[ Post a follow-up to this message ]

rich.teer@rite-group.com wrote...
> On Wed, 15 Dec 2004, Vendor Neutral wrote:
> 
>
> Why do you think that?

Mmmmm...because I'm a dope?

And, being a dope, I was concerned about the latency of the
transition from sleep to getting placed in the run queue, when there
are data waiting.
 
>
> Umm, if there is data available, blocking functions won't block: they'll
> return data immediately.
>
> Using functions that block if no data is available is the correct
> thing to do.

[ Post a follow-up to this message ]

On Wed, 15 Dec 2004, Vendor Neutral wrote:

> Mmmmm...because I'm a dope?

The fact that you asked the question suggests otherwise.  :-)

> And, being a dope, I was concerned about the latency of the
> transition from sleep to getting placed in the run queue, when there
> are data waiting.

That would only be an issue on a R E A L L Y heavily loaded system,
in which case, you've got other problems, and probably wouldn't be
able to process the data in a timely fashion anyway.

Any reasonably modern machine can process packets from a fully
saturated 100 baseT interface.  And processing data as fast as
the wire can get it to you is the best you can do!

--
Rich Teer, SCNA, SCSA, author of "Solaris Systems Programming"

President,
Rite Online Inc.

Voice: +1 (250) 979-1638
URL: http://www.rite-group.com/rich

[ Post a follow-up to this message ]

Vendor Neutral <neutral@domain.invalid> writes:

>Mmmmm...because I'm a dope?

>And, being a dope, I was concerned about the latency of the
>transition from sleep to getting placed in the run queue, when there
>are data waiting.

You should be more worried about your application burning CPU cycles
your network driver may need to process incoming packets.

Casper
--
Expressed in this posting are my opinions.  They are in no way related
to opinions held by my employer, Sun Microsystems.
Statements on Sun products included here are not gospel and may
be fiction rather than truth.

[ Post a follow-up to this message ]

rich.teer@rite-group.com wrote...
> On Wed, 15 Dec 2004, Vendor Neutral wrote:
> 
>
> The fact that you asked the question suggests otherwise.  :-)
> 
>
> That would only be an issue on a R E A L L Y heavily loaded system,
> in which case, you've got other problems, and probably wouldn't be
> able to process the data in a timely fashion anyway.
>
> Any reasonably modern machine can process packets from a fully
> saturated 100 baseT interface.  And processing data as fast as
> the wire can get it to you is the best you can do!

I switched to blocking versions of msgsnd() and msgrcv() (by removing
the IPC_NOWAIT flag and adjusting < 0 handling to deal with EIDRM and
EINTR errno returns) and I am literally stunned at the reduction in
CPU across the board.  For example, from 90+% to 0.7% on my second
daemon, with no loss of throughput.

Man, what an education I just got.

Many, many thanks!

[ Post a follow-up to this message ]

Casper.Dik@Sun.COM wrote...
> Vendor Neutral <neutral@domain.invalid> writes:
> 
> 
>
> You should be more worried about your application burning CPU cycles
> your network driver may need to process incoming packets.

That, of course, is an extremely good point.

[ Post a follow-up to this message ]

On Wed, 15 Dec 2004, Vendor Neutral wrote:

> Man, what an education I just got.
>
> Many, many thanks!

You're welcome.

You might want to have a look at my book, Solaris Systems Programming.*
Despite the word "Solaris" in its title, most of the material applies
to most UNIX and UNIX-like OSes.

The book's home page, www.rite-group.com/rich/ssp contains a lot of
info about it, including a sample chapter.

* Or preferably, buy a copy!  :-)

--
Rich Teer, SCNA, SCSA, author of "Solaris Systems Programming"

President,
Rite Online Inc.

Voice: +1 (250) 979-1638
URL: http://www.rite-group.com/rich

[ Post a follow-up to this message ]

rich.teer@rite-group.com wrote...
> On Wed, 15 Dec 2004, Vendor Neutral wrote:
> 
>
> You're welcome.
>
> You might want to have a look at my book, Solaris Systems Programming.*
> Despite the word "Solaris" in its title, most of the material applies
> to most UNIX and UNIX-like OSes.
>
> The book's home page, www.rite-group.com/rich/ssp contains a lot of
> info about it, including a sample chapter.
>
> * Or preferably, buy a copy!  :-)

Ordered via your link w/Amazon.  Merry Christmas to me.

Thanks again, Rich.  Happy holidays.

[ Post a follow-up to this message ]