f



Choices, choices, choices ...

Lectoribus Salutem,

I was directed to this forum from comp.arch by one of your regulars, Joe 
Seigh, because of the following question:

Given that you want to have a RA*S* (i.e., Reliable, Available and 
*Scalable* OS), how would you perform the following task on a ~100 CPU 
SMP machine:

"Measure the number of instances of <some> kernel event"

1. You create a system wide single counter protected by a single
    system wide lock to keep track of the occurences of said event.

2. You create a processor-local counter of said event, and, in case
    you want to summarize system wide, sum the counters in the user
    space application and present them to the user.

Please, discuss.

Note that I'm not a regular of your newsgroup, so it's advisable to cc 
me if you have an answer different from the obvious ....

-- 
Toon Moene - e-mail: toon@moene.indiv.nluug.nl - phone: +31 346 214290
Saturnushof 14, 3738 XG  Maartensdijk, The Netherlands
Maintainer, GNU Fortran 77: http://gcc.gnu.org/onlinedocs/g77_news.html
A maintainer of GNU Fortran 95: http://gcc.gnu.org/fortran/
0
Toon
9/29/2004 11:37:25 PM
comp.programming.threads 4878 articles. 1 followers. Post Follow

6 Replies
1168 Views

Similar Articles

[PageSpeed] 51

> 2. You create a processor-local counter of said event, and, in case
>    you want to summarize system wide, sum the counters in the user
>    space application and present them to the user.

This would be one way to go. The contention on a single system wide lock 
would be very bad.

If the counter is small enough to be modified by atomic ops... I would use a 
padded and cache-line aligned counter per-cpu, and modify them with atomic 
inc/dec or cas. 


0
SenderX
9/30/2004 12:16:54 AM

Toon Moene wrote:
> 
> Lectoribus Salutem,
> 
> I was directed to this forum from comp.arch by one of your regulars, Joe
> Seigh, because of the following question:
> 
> Given that you want to have a RA*S* (i.e., Reliable, Available and
> *Scalable* OS), how would you perform the following task on a ~100 CPU
> SMP machine:
> 
> "Measure the number of instances of <some> kernel event"
> 
> 1. You create a system wide single counter protected by a single
>     system wide lock to keep track of the occurences of said event.
> 
> 2. You create a processor-local counter of said event, and, in case
>     you want to summarize system wide, sum the counters in the user
>     space application and present them to the user.
> 
> Please, discuss.

The distributed count would be more efficient if the reads are relatively
infrequent because reading a distributed counter is fairly expensive. Statistical
counters are typically kept in processor local storage.

You must have hit a syscall that escaped everyone's attention because
the kernel performance people would have been all over the kernel developers
for using a lock that way on a 100 processor system.

There could of course be reasons we don't know about.  If the thread was
preemptable you wouldn't want to increment a counter that way, you might
end up storing the count + 1 into another processors count which would
be bad.  Still, there's no excuse for using a performance killing solution.
There's ways around the problem even if you're preemptable.

File a performance problem report.  You're the customer.

Joe Seigh
0
Joe
9/30/2004 12:54:19 AM
In article <415b45b7$0$283$4d4ebb8e@news.nl.uu.net>,
 Toon Moene <toon@moene.indiv.nluug.nl> wrote:

> Lectoribus Salutem,
> 
> I was directed to this forum from comp.arch by one of your regulars, Joe 
> Seigh, because of the following question:
> 
> Given that you want to have a RA*S* (i.e., Reliable, Available and 
> *Scalable* OS), how would you perform the following task on a ~100 CPU 
> SMP machine:
> 
> "Measure the number of instances of <some> kernel event"
> 
> 1. You create a system wide single counter protected by a single
>     system wide lock to keep track of the occurences of said event.
> 
> 2. You create a processor-local counter of said event, and, in case
>     you want to summarize system wide, sum the counters in the user
>     space application and present them to the user.

Well, it depends on how often the event occurs.  And (2) is rife with 
assumptions. (like either being on a non-preemptable kernel or disabling 
preemption/interrupts, and that the summation has to occur in userland).

To add some real-life examples from Solaris:

1.  memory allocations and frees are tracked by kmem cache (~200 in a 
stock system.  general-purpose allocations come from 32 of them, with 
sizes roughly exponential)  Each cache has a set of per-CPU buckets, 
each protected by a lock.  There are two 64-bit counters protected by 
the lock, one for allocations, one for frees.  The locks allow the 
kernel to do allocations without worrying about preemption, etc.  (full 
details are at http://www.usenix.org/event/usenix01/bonwick.html + 
http://www.usenix.org/publications/library/proceedings/bos94/bonwick.html
)

When you want allocation statistics for a cache:

        % kstat -n kmem_alloc_8

a kernel routine sums the numbers from each per-CPU bucket and reports 
the total to userland.


2.  CPU interrupt counts and elapsed times are tracked per-cpu, 
per-level, since the counting code for level N can be interrupted by a 
level N+1 interrupt.

        % kstat -m cpu -n intrstat

These don't have to worry about preemption, of course.

3.  There are a whole set of CPU-specific counts (context-switches, etc.)

        % kstat -m cpu -n sys
        % kstat -m cpu -n vm

These are handled by disabling preemption, bumping the count, then 
re-enabling preemption.


4.  DTrace (http://www.sun.com/bigadmin/content/dtrace/) widens the 
possibilities substantially.  For example, if I want to count the number
of calls to "kmem_alloc()" on all CPUs in a scalable fashion, all I need
to do is:

        # dtrace -q -n 'kmem_alloc:entry{@a = count()}' \
            -n 'END{printa("%@d calls\n", @a)}' 
        ... wait a while ...
        ^C
        96665 calls
        #

DTrace automatically handles instrumenting the function, doing the 
counting in per-CPU buffers, etc.


As usual, there are quite a few engineering trade-offs in the mix.

Cheers,
- jonathan
0
Jonathan
9/30/2004 12:56:41 AM
In article <WfI6d.133353$MQ5.70575@attbi_s52>, "SenderX" <xxx@xxx.com> 
wrote:

> > 2. You create a processor-local counter of said event, and, in case
> >    you want to summarize system wide, sum the counters in the user
> >    space application and present them to the user.
> 
> This would be one way to go. The contention on a single system wide lock 
> would be very bad.
> 
> If the counter is small enough to be modified by atomic ops... I would use a 
> padded and cache-line aligned counter per-cpu, and modify them with atomic 
> inc/dec or cas. 

Since we are talking about kernels, a much faster approach is to put 
your counters in the per-CPU structure the kernel should already *have*, 
and only allow modifications to them from that CPU.  So a counter
bump becomes:

        /* disable preemption, if needed */
        CPU->my_counter++;
        /* re-enable preemption, if needed */

This avoids atomic ops, bus locking, etc., and has quit low overhead.

Cheers,
- jonathan
0
Jonathan
9/30/2004 1:08:10 AM
In article <415B59AB.2FBB9441@xemaps.com>,
 Joe Seigh <jseigh_01@xemaps.com> wrote:

> The distributed count would be more efficient if the reads are relatively
> infrequent because reading a distributed counter is fairly expensive. 
> Statistical counters are typically kept in processor local storage.

And statistical counts are usually very rarely read.

> You must have hit a syscall that escaped everyone's attention because
> the kernel performance people would have been all over the kernel developers
> for using a lock that way on a 100 processor system.

Another possibility is that a design decision that made sense when it 
was made N years ago was invalidated by changes elsewhere -- for example,
making an instruction which used to be implemented in hardware go into a 
trap handler on a newer processor could cause a trap handler to be hit 
much harder than the original design expected it to be.

There tend to be unexpected shifts in the trade-offs, sometimes drastic, 
as time goes on.  And a kernel tends to be a large beast -- until 
someone notices the problem, performance issues can hide for a long time.
 
> There could of course be reasons we don't know about.  If the thread was
> preemptable you wouldn't want to increment a counter that way, you might
> end up storing the count + 1 into another processors count which would
> be bad.  Still, there's no excuse for using a performance killing solution.
> There's ways around the problem even if you're preemptable.

Disabling and enabling preemption is typically extremely cheap -- bump a 
per-thread counter, <pre-emption disabled> decrement it, if it's zero 
check a separate "special processing" flag.

> File a performance problem report.  You're the customer.

Indeed.

- jonathan
0
Jonathan
9/30/2004 1:31:29 AM

Jonathan Adams wrote:
> 
> In article <415B59AB.2FBB9441@xemaps.com>,
>  Joe Seigh <jseigh_01@xemaps.com> wrote:
> 

> > There could of course be reasons we don't know about.  If the thread was
> > preemptable you wouldn't want to increment a counter that way, you might
> > end up storing the count + 1 into another processors count which would
> > be bad.  Still, there's no excuse for using a performance killing solution.
> > There's ways around the problem even if you're preemptable.
> 
> Disabling and enabling preemption is typically extremely cheap -- bump a
> per-thread counter, <pre-emption disabled> decrement it, if it's zero
> check a separate "special processing" flag.
> 

schedctl for the kernel.  And of course you can depend on it being mandatory
unlike user space.  The count trick is neat.  It's not interruptable once
you set it to one.

Joe Seigh
0
Joe
9/30/2004 1:56:07 AM
Reply:

Similar Artilces:

Choices, choices, choices
The battle has raged for years, inside the sacred virtual halls of CSMA and beyond, over who has the best computer platform. It's time to stop the silliness. Choices, we all have to make them. Even more fundamentally, we are making choices every moment. By reading this post, you are making a choice. The brand of computer and the software you are using are also choices. Every aspect of your life that led to the present moment is the consequence of past choices. Choice is even more certain than death and taxes, those after all also represent choices we make. (Yes, death is a choice, a...

Time for me to upgrade... Choices, choices, choices??
Hi there all you tech heads. I am after advice from "those in the know" about a box upgrade that I'm considering. Having being through this once before (and fluffing it with a 733 P3 just after P4 release) and missing any opportunity to make a meaningful (cpu only) upgrade. I "NEED" to spend as little as possible on my initial upgrade of Case, m/b, cpu (low budget), 512mb ddr & hdd. I have a reasonable appreciation of the value of higher fsb, P4 hyper threading and 64 bit 754 XP's, but don't have much of an idea as to the comparative value of the 462 XP...

[News] The Choice of Having No Choice
Microsoft's Ballmer preaches gospel of the rich client ,----[ Quote ] | With Microsoft and Intel we are the holy trinity - Jerry Sanders III, | ex-CEO of AMD | | [...] | | Microsoft is a rich server and Intel and AMD are rich clients. The | rest of us are unlikely to be enriched by their "innovations" | because pretty soon if you want to buy a PC you're going to be | practically forced to buy one with Vista, and with dual cores, | quadruple cores and the rest, simply because the vendors, the | distribution chain, the ISVs and the ODMs are all firmly pledged | to keep thei...

Choices$Write,Choices$Path
From Justin >Choices$Write is, generally, set once. Together with Choices$Path it >provides your configuration settings. If they're changed together (or even >independently) then your application won't see the files it saw before, >but it also should be robust and just use the new area. It might not be a >common operation, but you support it because your program is resillient to >such changes. > >[snip] > >> By using a primary Choices variable Myprog$Choices, the user could >> drag the choices destination to RAM or Memphis or set...

OT: Is any choice harder than no choice?
Recently a poster claimed that 'Adding a second choice makes it "harder for a consumer to choose'. " Does anyone have any scientific evidence to back that up? Short term memory is linked to decision making, and it is known that a human can have around 3-4 things in short term memory at a given time, so it seems to me that adding choice to the mixture would not make it harder. I have referenced plenty of links to studies that seemingly back this up (although to what extent is under dispute). So does anyone have any actual quotes or links to back up that a single choice is hard...

logic of limiting choices and forcing choices
Hi, I just can not find the set based sql approach to the following application logic: I have a table of choices with id and text, say: 1 coffee 2 tea 3 water 4 juice Now some users have restrictions, for example user 1 is disallowed coffeine. Some Users are forced one choice - user 2 is only allowed water, so there is another type of restriction. This would yield table choicerestrictions (userid, choiceid, tor): 1 2 exclude 1 2 exclude 2 3 force Is this a known application pattern I can read about somewhere? How could I get a result set in one select? This would be...

Choice
In the grocery store there are many, many kinds of cheese... I have tried several but many I have not and I have little interest in them. I do, however, sometimes make my own cheese. I just had a snack with some of my home made cream cheese. It was excellent. Choice. Grocery stores also have many kinds of pancake mix. I do not believe I have ever bought any of them, though I have had others pancakes made from them. I make my own. Started by - for the most part - following recipes. Then I made my own recipe... one very different from any other I have seen. I shared it with others... m...

Choices
I mentioned humans as echoing machines. The following is an addition to that thought. In the process from input to output we have choices. Choice 1: absorption Choice 2: reflection With choice 1 something might change. With choice 2 nothing changes. Of course those processes are mingled. I'm interested in the changes. Example: input is a signal that says make zero one. Absorption: a zero changes to one or a one remains a one Reflection: a zero remains a zero. In addition I recall the echo where a one becomes a zero. Important is the notion of resistence. If a dumb machine just chan...

Few Choices
I installed Tiger on my iBook 800 mHz G3 while the iBook was in Target Mode (it has no DVD drive). During the installation process, i was only given two choices "Custom" install or "Upgrade." What's with that? I thought I would have "Archive & Install" and the other choices. Was there a problem happening? Ideas? ...

Choices
I am new to freeBSD. What should I consider using for web server work, 4.x or 5.x. What are the advantages of on over the other? Thanks Deke wrote: > I am new to freeBSD. What should I consider using for web server work, > 4.x or 5.x. What are the advantages of on over the other? Choose 5.3. If you find that something doesn't work (ie. the hardware doesn't work, or some software don't work), then try 4.10. 4.x is the "old technology" stable branch. Everything that works there is good and stable. 5.x is the "new technology" branch. T...

Browser choice should also extend to OS choice
Browser choice should also extend to OS choice ---------------------------------------------- The EU has ruled that its citizens should have browser choice because micoshaft had behaved as an illegal monopoly, and that choice quickly allowed many other browsers to go mainstream. The same is overdue for OS. Micoshaft has illegally twisted suppliers in Taiwan to restrict choice of OS preload. What used to be plentiful cheap netbooks with Linux has been replaced with ugly slow booting windummy OSen on machines that cannot properly run that OSen. Linux on the other hand can. So the EU are still...

What are my choices?
I'm in the middle of building a database in access 2002 for a client of mine. I just wanted to know what my choices were to make this database accessable from multiple clients at once. His network is a windows 2000 domain with about 8 workstations that will use this database. I read about an MDE file, but I when i tried to make it froze up my system. What are my choices? ricky@rickybell.com (InDeSkize) wrote in message news:<22935d4a.0405271156.75bbb5ca@posting.google.com>... > I'm in the middle of building a database in access 2002 for a client > of mine....

Choice
I don't know where to take a good also objective answer...so I ask it here... I use linux for years, big fun... I use it for office, multimedia, and some c++ small projects... office for economical studies multimedia like writing dvd and playing xvids after having transcoded the dvds Games.Yes in Linux. BUT. I saw OpenSolaris. Can it do it? Better than linux in performance? I start to hate the performance of modern distros like, fedora or...mandriva...either they won't work or they work with all the -g in gcc and various misworks.Not to mention the libxxx problem......

My choice..
Hi I'm a Korean.. I strongly recommend a Toshiba Laptop - Tecra series. That is a so good, powerful, and pretty... *^_^* ... You will have a great contentment. good luck!! ...

[News] GNU/Linux as the Choice for Those Who Love Choice
-----BEGIN PGP SIGNED MESSAGE----- Hash: SHA1 Windows: Choice But No Choice ,----[ Quote ] | In the area of window managers Linux users | are completely and totally spoiled rotten. We | constantly debate the merits of one desktop | environment/window manager over another. We | argue over what programs are better than | others, what versions of those programs we | like over another and getting in world class | pissing contests all the while crying about | what we wish they would do better or | differently. I wish Windows users had this | problem, but they don't. Why? Because the...

What are my choices?
I'm in the middle of building a database in access 2002 for a client of mine. I just wanted to know what my choices were to make this database accessable from multiple clients at once. His network is a windows 2000 domain with about 8 workstations that will use this database. I read about an MDE file, but I when i tried to make it froze up my system. What are my choices? The first and most important this is to split the front-end and the back-end of the database: Simplefied Concept: back-end (BE) = tables, design does not change front-end (FE) = everything else (code, forms, queries,...

c++ as choice for long term application choice.
Have a application (custom written for a vetical market) which has been developed in VB over the last several years. Now that MS has done what MS does best and is to relegated VB to the twilight zone in feb of 08 I have decided to rewrite in another language. The application is constatnly being upgraded with new features and I would like to find someting that would have long term support, if such is possible. Never would have thought MS would "disregard" VB. How about VC++. ? It would need to be somewhat on par with VB as far as the features and capabilities of VB. Any thoughts fro...

Choices !!
Hi, I have a JVC mini DV camcorder, and I like to be able to edit the footage on my pc a bit - add a few transitions, titles etc....... I like to burn the footage to a VCD with menus etc..... (my DVD player doesnt play SVCDs) Sometimes I may want to burn onto DVD -R or DVD-RAM....... With this in mind, what are the best settings to have Video Studio 7 setup to ? Should the PROJECT SETTINGS be set to AVI, 25fps, before I start ? Then when I capture, should I capture in DV format ? or MPEG, or AVI ? Am I right in thinking that I should try and capture in the format b...

selecting a subset of choices based on a first choice
Hello, I having trouble finding information on what I would like to do in Access probably because I do not know the right keywords. I am designing a that uses genus and species of plants in two of the fields (names genus and species). Since there is a limited number of species for a given genus, I would like the second field on the form have only the values that match that genus. For example, if my table looks like this: genus1, species1 genus1, species2 genus1, species3 genus2, species4 I would like to make it possible that if you select genus1, the next fields pull down menu would onl...

Quiz in VB
Can anyone please give me some clues on how to track the right answers in multiple-choice type questions (using checkbox) and single-choice questions (rdio buttons) and also how to store the right answer value in the db? Much appreciated kadmazid@hotmail.com (Mohammed Mazid) wrote in message news:<7cfd7b4a.0402140837.5a8f9570@posting.google.com>... > Can anyone please give me some clues on how to track the right answers > in multiple-choice type questions (using checkbox) and single-choice > questions (rdio buttons) and also how to store the right answer value > in the db? ...

Choice of DHCP-server? Is the "IOS-one" a good choice?
Have a need for a DHCP server. Is the IOS-one a good choice? The arguments for is: - Easy management and logging - HA, if the IOS-box is protected with UPS etc - No more boxes/computers Reasons why not: - possible security risks(eg IOS DHCP on main router in network) ?? - lack of functionality (option 82 etc)? - etc? I am not an expert on DHCP solutions so what du you reckon? /Fred "Fred" <Xfrecarlen@hotmail.com> wrote in message news:HLKBb.37586$mU6.133488@newsb.telia.net... > Reasons why not: > - possible security risks(eg IOS DHCP on main router in network) ??...

[c.d.o.s][Long...] SHMMNI choice affect SHMMAX choice?
People, I've been asked to document how a DBA should set values of some Linux kernel parameters. This is an easy task since it is a topic covered in the Oracle installation documents. While writing my document, my curiosity triggered some simple questions. What is the purpose of SHMMNI? If I set SHMMNI to a larger value, can I lower SHMMAX to a value to support a given SGA? I found this description of SHMMNI via google: SHMMNI sets the maximum number of shared memory segments which can be allocated at any one time. The above description suggests that I can carve up my shared memory...

Can Choice components respond to keyboard input like HTML Choice components?
Users expect Choice components to respond to keyboard input similarly to HTML Choice components. This keyboard-responsiveness is not coded into standard Java Choice components. One would expect that adding appropriate listeners and events could produce the HTML-like functionality, but this does not appear to work. At http://segal.org/java/ChoiceKey2/index.html is a small working applet with full source code to demonstrate this problem. The desired behavior occurs with neither the Sun JVM nor the Microsoft JVM; each departs from user expectations in different ways. Is there some way to do ...

Programming language of choice?
For building Windows desktop apps, the clear favorite is C#. But my clients can't afford to buy Microsoft products. So I need to develop software for Linux users and web applications. In the open source world, what is the programmer's language of choice? Judging by the number of members in each of these http://www.google.com/Top/Computers/Programming/Languages/Open_Source/ user groups, it looks like the top 3 open source languages are: 1. Java 2. PHP 3. Perl Does this mean Java is the open source language of choice? Are Java apps compiled, or is it an interpreted language? What...

Web resources about - Choices, choices, choices ... - comp.programming.threads

Choice - Wikipedia, the free encyclopedia
... style . The references used may be made clearer with a different or consistent style of citation , footnoting , or external linking . Choice ...

Negative gearing changes would push up rents: Mortgage Choice
The coalition's proposals to cap negative gearing for housing investors will restrict the supply of rental properties and push up rents, says ...

Barack Obama told Ellen DeGeneres that being president is all about making 'big choices'
President Barack Obama recently went on “The Ellen DeGeneres Show.” During his last year as president, he reflected on his first. “The good ...

The Volkswagen GTI was our daily driver of choice
Filed under: Volkswagen , Long-Term Garage , Hatchback , Performance We discuss what we learned from a long-term test of a 2015 Volkswagen GTI. ...

Pro-choice economics #ghostabortions
Pro-choice economics by digby There's a lively discussion going on on the left about sexism and the relationship of such problems to economics. ...

Barack Obama told Ellen DeGeneres that being president is all about making 'big choices'
President Barack Obama recently went on "The Ellen DeGeneres Show." During his last year as president, he reflected on his first. "The good ...

Nitin Desai: Budget choices - Business Standard Column New Document Widgets Magazine Widgets Magazine ...
The finance minister faces a difficult problem in choosing between two sets of advisers. Both factions accept that the Budget must aim at stimulating ...

Poll: South Carolina voters making top choices clear
New CBS News Battleground Tracker shows which Democratic and Republican candidate are in control in key primary

Australia Telecommunications Firm To Provide Customers Solar & Storage Choices
... trend. “Solar-based generation has been available [&hellip Australia Telecommunications Firm To Provide Customers Solar & Storage Choices was ...

Ki and Ka: Why Arjun Kapoor's gutsy choice is a benchmark for other Gen-Y actors!
Ki and Ka: Why Arjun Kapoor's gutsy choice is a benchmark for other Gen-Y actors! - When Arjun Kapoor decided to play the 'house husband' in ...

Resources last updated: 2/18/2016 10:49:21 PM