Typical problem where a while(1) consumes gobs of CPU.  Not as 
straightforward (to me) to figure out how to replace it with 
something more CPU friendly.

The overall system has three daemons:

1) suck_packets_d
2) first_order_packet_filter_and_packet_buffer_d
3) extract_interesting_packet_features_and_generate_report_d

Using some sort of blocking calls in 1) or 2) seems like a bad idea, 
because packet volume can be high and I want to keep up with it as 
much as possible.  

I could use some help thinking this through if anyone is so inclined.

C / Linux

pseudocode for daemon (2) (biggest CPU consumer)

main()
{
	while(1)
	{
		retcode = function() ;
		switch retcode 
		{
			case	errortype1
			case	errortype2
			case	errortype3
		}
	}
	
	return 0 ;
}

int function()
{
	value = 0 ;

	// value is pass-by-result here
	read_incoming_svs_v_message_queue( value ) ; 

	if keep_new_data( value )
		return write_outgoing_sys_v_message_queue() ;
	else
		return	NO_OP ;	
}

int read_incoming_svs_v_message_queue( value ) 
{
	if msgrcv() < 0
		if errno != ENOMSG
			return ERROR
		else
			return	NO_OP
	else
	{
		read_inbound_posix_shared_memory() ;
		
		while (1)
		{
			if fcntl() GET LOCK < 0
			{
				if errno == EAGAIN or errno == EACCES
					continue ;
				else 
					return	ERROR
			}
		}

		value = read locked resource

		if fcntl() UNLOCK < 0 
			handle_error()		
	}
}

int write_outgoing_sys_v_message_queue()
{
	while (1)
	{
		if fcntl() GET LOCK < 0
		{
			if errno == EAGAIN or errno == EACCES
				continue ;
			else 
				return	ERROR
		}
	}

	write to locked resource

	if fcntl() UNLOCK < 0 
		handle_error()

	if msgsnd() fails
		return	ERROR ;

	write to posix shared memory 
}

"Vendor Neutral" <neutral@domain.invalid> wrote in message
news:MPG.1c2a4f80ebc7ecbf989f9f@news.verizon.net...
> Typical problem where a while(1) consumes gobs of CPU.  Not as
> straightforward (to me) to figure out how to replace it with
> something more CPU friendly.
>
> The overall system has three daemons:
>
> 1) suck_packets_d
> 2) first_order_packet_filter_and_packet_buffer_d
> 3) extract_interesting_packet_features_and_generate_report_d
>
> Using some sort of blocking calls in 1) or 2) seems like a bad idea,
> because packet volume can be high and I want to keep up with it as
> much as possible.

Sorry, but "blocking calls" are designed for this purpose.  Why is it a bad
idea?  If the data is there to process then you are unblocked.  These types
of semaphores are generally designed to work with the I/O and CPU to create
the best possible scenario.

On Wed, 15 Dec 2004, Vendor Neutral wrote:

> Using some sort of blocking calls in 1) or 2) seems like a bad idea,

Why do you think that?

> because packet volume can be high and I want to keep up with it as
> much as possible.

Umm, if there is data available, blocking functions won't block: they'll
return data immediately.

Using functions that block if no data is available is the correct
thing to do.

-- 
Rich Teer, SCNA, SCSA, author of "Solaris Systems Programming"

President,
Rite Online Inc.

Voice: +1 (250) 979-1638
URL: http://www.rite-group.com/rich

rich.teer@rite-group.com wrote...
> On Wed, 15 Dec 2004, Vendor Neutral wrote:
> 
> > Using some sort of blocking calls in 1) or 2) seems like a bad idea,
> 
> Why do you think that?

Mmmmm...because I'm a dope?

And, being a dope, I was concerned about the latency of the 
transition from sleep to getting placed in the run queue, when there 
are data waiting.

> > because packet volume can be high and I want to keep up with it as
> > much as possible.
> 
> Umm, if there is data available, blocking functions won't block: they'll
> return data immediately.
> 
> Using functions that block if no data is available is the correct
> thing to do.

On Wed, 15 Dec 2004, Vendor Neutral wrote:

> Mmmmm...because I'm a dope?

The fact that you asked the question suggests otherwise.  :-)

> And, being a dope, I was concerned about the latency of the
> transition from sleep to getting placed in the run queue, when there
> are data waiting.

That would only be an issue on a R E A L L Y heavily loaded system,
in which case, you've got other problems, and probably wouldn't be
able to process the data in a timely fashion anyway.

Any reasonably modern machine can process packets from a fully
saturated 100 baseT interface.  And processing data as fast as
the wire can get it to you is the best you can do!

-- 
Rich Teer, SCNA, SCSA, author of "Solaris Systems Programming"

President,
Rite Online Inc.

Voice: +1 (250) 979-1638
URL: http://www.rite-group.com/rich

Vendor Neutral <neutral@domain.invalid> writes:

>Mmmmm...because I'm a dope?

>And, being a dope, I was concerned about the latency of the 
>transition from sleep to getting placed in the run queue, when there 
>are data waiting.

You should be more worried about your application burning CPU cycles
your network driver may need to process incoming packets.

Casper
-- 
Expressed in this posting are my opinions.  They are in no way related
to opinions held by my employer, Sun Microsystems.
Statements on Sun products included here are not gospel and may
be fiction rather than truth.

rich.teer@rite-group.com wrote...
> On Wed, 15 Dec 2004, Vendor Neutral wrote:
> 
> > Mmmmm...because I'm a dope?
> 
> The fact that you asked the question suggests otherwise.  :-)
> 
> > And, being a dope, I was concerned about the latency of the
> > transition from sleep to getting placed in the run queue, when there
> > are data waiting.
> 
> That would only be an issue on a R E A L L Y heavily loaded system,
> in which case, you've got other problems, and probably wouldn't be
> able to process the data in a timely fashion anyway.
> 
> Any reasonably modern machine can process packets from a fully
> saturated 100 baseT interface.  And processing data as fast as
> the wire can get it to you is the best you can do!

I switched to blocking versions of msgsnd() and msgrcv() (by removing 
the IPC_NOWAIT flag and adjusting < 0 handling to deal with EIDRM and 
EINTR errno returns) and I am literally stunned at the reduction in 
CPU across the board.  For example, from 90+% to 0.7% on my second 
daemon, with no loss of throughput.

Man, what an education I just got.

Many, many thanks!

Casper.Dik@Sun.COM wrote...
> Vendor Neutral <neutral@domain.invalid> writes:
> 
> >Mmmmm...because I'm a dope?
> 
> >And, being a dope, I was concerned about the latency of the 
> >transition from sleep to getting placed in the run queue, when there 
> >are data waiting.
> 
> You should be more worried about your application burning CPU cycles
> your network driver may need to process incoming packets.

That, of course, is an extremely good point.

On Wed, 15 Dec 2004, Vendor Neutral wrote:

> Man, what an education I just got.
>
> Many, many thanks!

You're welcome.

You might want to have a look at my book, Solaris Systems Programming.*
Despite the word "Solaris" in its title, most of the material applies
to most UNIX and UNIX-like OSes.

The book's home page, www.rite-group.com/rich/ssp contains a lot of
info about it, including a sample chapter.

* Or preferably, buy a copy!  :-)

-- 
Rich Teer, SCNA, SCSA, author of "Solaris Systems Programming"

President,
Rite Online Inc.

Voice: +1 (250) 979-1638
URL: http://www.rite-group.com/rich

rich.teer@rite-group.com wrote...
> On Wed, 15 Dec 2004, Vendor Neutral wrote:
> 
> > Man, what an education I just got.
> >
> > Many, many thanks!
> 
> You're welcome.
> 
> You might want to have a look at my book, Solaris Systems Programming.*
> Despite the word "Solaris" in its title, most of the material applies
> to most UNIX and UNIX-like OSes.
> 
> The book's home page, www.rite-group.com/rich/ssp contains a lot of
> info about it, including a sample chapter.
> 
> * Or preferably, buy a copy!  :-)

Ordered via your link w/Amazon.  Merry Christmas to me.

Thanks again, Rich.  Happy holidays.

On Wed, 15 Dec 2004, Vendor Neutral wrote:

> Ordered via your link w/Amazon.  Merry Christmas to me.

Thank you!

> Thanks again, Rich.  Happy holidays.


My pleasure - and Merry Christmas to you, too.

WHich reminds me--it's way past the time I should dig out
my Xmas signature.  I'll rectify that right now!

-- 
Rich Teer, SCNA, SCSA, author of "Solaris Systems Programming"

                                                    .  *   * . * .* .
                                                     .   *   .   .*
President,                                          * .  . /\ ( .  . *
Rite Online Inc.                                     . .  / .\   . * .
                                                    .*.  / *  \  . .
                                                      . /*   o \     .
Voice: +1 (250) 979-1638                            *   '''||'''   .
URL: http://www.rite-online.net                     ******************