f



How to write a kernel patch to expose a kernel-level function?

I want to be able to call smp_processor_id() from user-space.  Of
course the patch would be used by only me.  (I am not asking someone to
write such to submit as an official patch.)

0
hzmonte (100)
1/30/2006 6:11:36 PM
comp.os.linux.misc 33536 articles. 0 followers. amosa69 (78) is leader. Post Follow

15 Replies
332 Views

Similar Articles

[PageSpeed] 46

hzmonte@hotmail.com writes:

> I want to be able to call smp_processor_id() from user-space.  Of
> course the patch would be used by only me.  (I am not asking someone to
> write such to submit as an official patch.)

Does this help:

http://altlinux.org/index.php?module=sisyphus&package=cpuid
0
daneNO2 (702)
1/30/2006 8:05:50 PM
hzmonte@hotmail.com wrote:
> I want to be able to call smp_processor_id() from user-space.  Of
> course the patch would be used by only me.  (I am not asking someone to
> write such to submit as an official patch.)


In principle, there are two ways of doing it:

  - write a new system call
  - create an entry in the /proc pseudo-filesystem

In both cases you need a piece of kernel code,
a patch or a kernel module.

Please be warned that the result processor may be
changed when you return from the system call reading
the information.

-- 

Tauno Voipio
tauno voipio (at) iki fi

0
1/30/2006 8:08:20 PM
>http://altlinux.org/index.php?module=sisyphus&package=cpuid
Thx but no, it is not what I want.  This package displays the CPU spec
like who the manufacturer is and what model it is, etc.  What I want,
and what smp_processor_id() returns, is which logical processor the
calling thread is running on.  Is CPU0, CPU1, or CPU2, for example?

>In both cases you need a piece of kernel code, a patch or a kernel module
So my question is How to write a kernel patch?  Is it as simple as
adding a statement in the source like "EXPORT TO USER-SPACE
smp_processor_id"?  Any pointer to how to do it?  Ay more concrete
guideline?

0
hzmonte (100)
1/31/2006 2:35:39 AM
hzmonte writes:
> What I want, and what smp_processor_id() returns, is which logical
> processor the calling thread is running on.  Is CPU0, CPU1, or CPU2, for
> example?

Why?  Of what possible use could that information be?  What problem are you
trying to solve?
-- 
John Hasler 
john@dhh.gt.org
Dancing Horse Hill
Elmwood, WI USA
0
john4584 (1601)
1/31/2006 2:52:33 AM
hzmonte@hotmail.com wrote:
>>http://altlinux.org/index.php?module=sisyphus&package=cpuid
> 
> Thx but no, it is not what I want.  This package displays the CPU spec
> like who the manufacturer is and what model it is, etc.  What I want,
> and what smp_processor_id() returns, is which logical processor the
> calling thread is running on.  Is CPU0, CPU1, or CPU2, for example?
> 
>>In both cases you need a piece of kernel code, a patch or a kernel module
> 
> So my question is How to write a kernel patch?  Is it as simple as
> adding a statement in the source like "EXPORT TO USER-SPACE
> smp_processor_id"?  Any pointer to how to do it?  Ay more concrete
> guideline?


The kernel runs in a private address space compared to
the user code. You cannot use the kernel internal
address for anything in user code, and it's not allowed
to even look at the contents of the address.

For details, get the books

   Understanding the Linux Kernel

and

   Linux Device Drivers

and read and understand them before attempting own
kernel coding.

Both books are published by O'Reilly.

IMHO, creating a /proc file is the easiest way of
reading the identifier.

A different story is that the result is probably
invalid at the time your user code gets it. There
is no guarantee that the process is running on
the reported processor anymore.

-- 

Tauno Voipio
tauno voipio (at) iki fi

0
1/31/2006 7:29:59 AM
>Why?  Of what possible use could that information be?  What problem are you
>trying to solve?

>A different story is that the result is probably invalid at the time your user code gets it. There
>is no guarantee that the process is running on the reported processor anymore.

I am trying to make sure sched_setaffinity() works.  That system call
allows you to restrict the execution of the calling thread on a (set
of) particular logical processor(s).  So, after I call
sched_setaffinity(), I want to call smp_processor_id() to see if the
thread is indeed executing on a processor selected.

0
hzmonte (100)
1/31/2006 8:03:38 AM
hzmonte@hotmail.com wrote:
>> Why?  Of what possible use could that information be?  What problem are you
>> trying to solve?
> 
>> A different story is that the result is probably invalid at the time your user code gets it. There
>> is no guarantee that the process is running on the reported processor anymore.
> 
> I am trying to make sure sched_setaffinity() works.  That system call
> allows you to restrict the execution of the calling thread on a (set
> of) particular logical processor(s).  So, after I call
> sched_setaffinity(), I want to call smp_processor_id() to see if the
> thread is indeed executing on a processor selected.
> 
Why not, instead, examine the kernel code related to this and see if it is
OK? Would that not be easier than tinkering with the kernel?

-- 
  .~.  Jean-David Beyer          Registered Linux User 85642.
  /V\  PGP-Key: 9A2FC99A         Registered Machine   241939.
 /( )\ Shrewsbury, New Jersey    http://counter.li.org
 ^^-^^ 05:15:00 up 10 days, 20:42, 5 users, load average: 4.08, 4.10, 4.09
0
jeandavid8 (984)
1/31/2006 10:19:25 AM
John Hasler <john@dhh.gt.org> wrote:
> hzmonte writes:
>> What I want, and what smp_processor_id() returns, is which logical
>> processor the calling thread is running on.  Is CPU0, CPU1, or CPU2, for
>> example?

> Why?  Of what possible use could that information be?  What problem are you
> trying to solve?

Dunno. Probably some mistaken idea. But anyway it would be simple to do
- some ioctl which returns what he wants. But doesn't he have the info
in proc anyway?

 % cat /proc/self/cpu
 cpu  0 1
 cpu0 0 1

Peter
0
ptb (2760)
1/31/2006 11:45:54 AM
Peter T. Breuer wrote:
> John Hasler <john@dhh.gt.org> wrote:
>> hzmonte writes:
>>> What I want, and what smp_processor_id() returns, is which logical
>>> processor the calling thread is running on.  Is CPU0, CPU1, or CPU2, for
>>> example?
> 
>> Why?  Of what possible use could that information be?  What problem are you
>> trying to solve?
> 
> Dunno. Probably some mistaken idea. But anyway it would be simple to do
> - some ioctl which returns what he wants. But doesn't he have the info
> in proc anyway?
> 
How would that do any good? When the process makes the kernel call on ioctl,
a process switch will occur and what would the ioctl return? The
processor-id that the process _used to_ run on, not the one it will run on
once the ioctl returns.

He is doomed. The only way I can think of that will work is to examine the
code of the kernel in the area where process switching occurs. I looked in
there a while ago, and it really tries to return a process to the same
processor it ran on before to try to maintain a high cache hit ratio, but
with the puny caches on my processors (1 Megabit L3 caches) compared to
process sizes (50Megabytes and up for some of the large ones), this may be
of only minor importance.

It depends on the working set size of all the processes currently running. I
have a dbms that runs many processes, and does a lot of IO. Runs pretty well
unless I let BOINC run 4 compute-limited processes at nice 19. Running BOINC
this way should take up all the otherwise unused CPU cycles and not
interfere with the higher priority processes, but it spoils the database
application(s). They try to do a disk write and suspend. Then a BOINC
process runs and dirtys up the caches, so when the IO for the dbms completes
and the dbms process resumes, the cache is completely dirty so it runs at
RAM speeds (533MHz) instead of processor speeds (3.06GHz).

-- 
  .~.  Jean-David Beyer          Registered Linux User 85642.
  /V\  PGP-Key: 9A2FC99A         Registered Machine   241939.
 /( )\ Shrewsbury, New Jersey    http://counter.li.org
 ^^-^^ 07:50:00 up 10 days, 23:17, 3 users, load average: 4.16, 4.11, 4.03
0
jeandavid8 (984)
1/31/2006 1:01:58 PM
Jean-David Beyer wrote:
> Peter T. Breuer wrote:
> > John Hasler <john@dhh.gt.org> wrote:
> >> hzmonte writes:
> >>> What I want, and what smp_processor_id() returns, is which logical
> >>> processor the calling thread is running on.  Is CPU0, CPU1, or CPU2, for
> >>> example?
> >
> >> Why?  Of what possible use could that information be?  What problem are you
> >> trying to solve?
> >
> > Dunno. Probably some mistaken idea. But anyway it would be simple to do
> > - some ioctl which returns what he wants. But doesn't he have the info
> > in proc anyway?
> >
> How would that do any good? When the process makes the kernel call on ioctl,
> a process switch will occur and what would the ioctl return? The
> processor-id that the process _used to_ run on, not the one it will run on
> once the ioctl returns.
>
> He is doomed. The only way I can think of that will work is to examine the
> code of the kernel in the area where process switching occurs. I looked in
> there a while ago, and it really tries to return a process to the same
> processor it ran on before to try to maintain a high cache hit ratio, but
> with the puny caches on my processors (1 Megabit L3 caches) compared to
> process sizes (50Megabytes and up for some of the large ones), this may be
> of only minor importance.
>
> It depends on the working set size of all the processes currently running. I
> have a dbms that runs many processes, and does a lot of IO. Runs pretty well
> unless I let BOINC run 4 compute-limited processes at nice 19. Running BOINC
> this way should take up all the otherwise unused CPU cycles and not
> interfere with the higher priority processes, but it spoils the database
> application(s). They try to do a disk write and suspend. Then a BOINC
> process runs and dirtys up the caches, so when the IO for the dbms completes
> and the dbms process resumes, the cache is completely dirty so it runs at
> RAM speeds (533MHz) instead of processor speeds (3.06GHz).

Isn't some of this info the OP desires (considering HT, and/or Dual
Core CPUs) highly dependent upon "which" brand (Intel or AMD64) the OP
is using ??

I'm not up on this at all - and I don't yet have a great grasp on
explaining the technology and functions , but doesn't AMD64 incorporate
_NUMA_, the HTT Bus, and the A64  Memory Controller is OnDie, ["MOESI
protocol" (Modified, Owned, Exclusive, Shared, Invalid)] since their ,
as oppposed to the Intel Architecture (Xeon, Pentium-D, and HT -- I'm
don't know about Itanium though).
<http://en.wikipedia.org/wiki/Non-Uniform_Memory_Access>
more info about the diffs;
<http://www.xbitlabs.com/articles/cpu/print/dual-core.html>

hmmm..wonder how MShafts PAE crap figures into all this too ?

0
1/31/2006 8:32:26 PM
> Why not, instead, examine the kernel code related to this and see if it is
OK? Would that not be easier than tinkering with the kernel?
Code examination is one way of making sure the code works as expected.
But I am not an expert at looking at kernel code. (It looks
intimidating.)  And who knows whether there are other code that may
affect the operation of the code I inspect. And if I can call that
smp_processor_id(), I can show it to others that the execution actually
occurs on a particular CPU as specified in sched_setaffinity() .  Also,
if I select a group of CPUs, rather than just one, in
sched_setaffinity(), code examination only tells me the execution will
happen on one of those CPUs, but smp_processor_id() tells me exactly
which one CPU is executing that thread.

0
hzmonte (100)
2/1/2006 2:29:35 AM
hzmonte@hotmail.com wrote:
>> Why not, instead, examine the kernel code related to this and see if it is
> OK? Would that not be easier than tinkering with the kernel?
> Code examination is one way of making sure the code works as expected.
> But I am not an expert at looking at kernel code. (It looks
> intimidating.)  And who knows whether there are other code that may
> affect the operation of the code I inspect. And if I can call that
> smp_processor_id(), I can show it to others that the execution actually
> occurs on a particular CPU as specified in sched_setaffinity() .  Also,
> if I select a group of CPUs, rather than just one, in
> sched_setaffinity(), code examination only tells me the execution will
> happen on one of those CPUs, but smp_processor_id() tells me exactly
> which one CPU is executing that thread.
> 
No. It tells you which one WAS executing that thread. But as the kernel
relinquishes the processor, it may give a different processor to you. Well,
if you locked yourself to one of them with some affinity trick, you would
get the same one back when it is available, but that is what you wanted to
find out.

In the short run, why not look at the output of top with the Y mode enabled.
Or look at the code for top (which I suppose is simpler than the kernel) and
see how they do it.

-- 
  .~.  Jean-David Beyer          Registered Linux User 85642.
  /V\  PGP-Key: 9A2FC99A         Registered Machine   241939.
 /( )\ Shrewsbury, New Jersey    http://counter.li.org
 ^^-^^ 23:05:00 up 11 days, 14:32, 5 users, load average: 4.37, 4.22, 4.12
0
jeandavid8 (984)
2/1/2006 4:10:02 AM
Jean-David Beyer <jeandavid8@verizon.net> wrote:
> Peter T. Breuer wrote:
>> John Hasler <john@dhh.gt.org> wrote:
>>> hzmonte writes:
>>>> What I want, and what smp_processor_id() returns, is which logical
>>>> processor the calling thread is running on.  Is CPU0, CPU1, or CPU2, for
>>>> example?
>> 
>>> Why?  Of what possible use could that information be?  What problem are you
>>> trying to solve?
>> 
>> Dunno. Probably some mistaken idea. But anyway it would be simple to do
>> - some ioctl which returns what he wants. But doesn't he have the info
>> in proc anyway?
>> 
> How would that do any good? When the process makes the kernel call on ioctl,
> a process switch will occur and what would the ioctl return? The

There is no special extra process switch on ioctl - he'll get whatever
cpu is running his ioctl.  Yes, it is another question if the kernel
decided to switch him to a different cpu on the i/o, but his chances of
that are low, or at least about as low as with anything he does if there
is any affinity at all. He can even change the schedule in his ioctl,
but that's harder.

> processor-id that the process _used to_ run on, not the one it will run on
> once the ioctl returns.

Indeed. But then the future is never predictable with certainty. The
real problem is that his experiment might provoke a change.

> He is doomed. The only way I can think of that will work is to examine the
> code of the kernel in the area where process switching occurs. I looked in
> there a while ago, and it really tries to return a process to the same
> processor it ran on before to try to maintain a high cache hit ratio, but
> with the puny caches on my processors (1 Megabit L3 caches) compared to
> process sizes (50Megabytes and up for some of the large ones), this may be
> of only minor importance.

One might be able to get general processor scheduler stats out on how
often a process is run by the same cpu.

> It depends on the working set size of all the processes currently running. I
> have a dbms that runs many processes, and does a lot of IO. Runs pretty well
> unless I let BOINC run 4 compute-limited processes at nice 19. Running BOINC
> this way should take up all the otherwise unused CPU cycles and not
> interfere with the higher priority processes, but it spoils the database
> application(s). They try to do a disk write and suspend. Then a BOINC
> process runs and dirtys up the caches, so when the IO for the dbms completes
> and the dbms process resumes, the cache is completely dirty so it runs at
> RAM speeds (533MHz) instead of processor speeds (3.06GHz).

Peter


0
ptb (2760)
2/1/2006 8:26:52 AM
Jean-David Beyer <jeandavid8@verizon.net> wrote:
> No. It tells you which one WAS executing that thread. But as the kernel
> relinquishes the processor, it may give a different processor to you. Well,
> if you locked yourself to one of them with some affinity trick, you would
> get the same one back when it is available, but that is what you wanted to
> find out.

http://www.daimi.au.dk/~kasperd/comp.os.linux.development.faq.html

  In kernel version 2.6 user mode processes can use the
  sched_setaffinity and sched_getaffinity system calls. There is a
  patch which backports them to 2.4.

  All recent Linux versions will try to keep processes as long time as
  possible on the same CPU. But processes will be moved if the CPUs are
  not equally loaded. True CPU affinity was introduced in 2.4.0, but is
  only available in kernel mode. It can be used from kernel modules,
  but was not used by the kernel itself. Starting in 2.4.7-pre5 it is
  used by ksoftirqd to start one instance for each CPU. In 2.5.8-pre3 a
  systemcall to set CPU affinity was introduced.

Peter
0
ptb (2760)
2/1/2006 11:02:54 AM
hzmonte@hotmail.com wrote:
> > Why not, instead, examine the kernel code related to this and see if it is
> OK? Would that not be easier than tinkering with the kernel?
> Code examination is one way of making sure the code works as expected.
> But I am not an expert at looking at kernel code. (It looks
> intimidating.)  And who knows whether there are other code that may
> affect the operation of the code I inspect. And if I can call that
> smp_processor_id(), I can show it to others that the execution actually
> occurs on a particular CPU as specified in sched_setaffinity() .  Also,
> if I select a group of CPUs, rather than just one, in
> sched_setaffinity(), code examination only tells me the execution will
> happen on one of those CPUs, but smp_processor_id() tells me exactly
> which one CPU is executing that thread.

FWIW -- RedHawk Linux
http://searchopensource.techtarget.com/specialEventsAnnouncement/0,289656,sid39_gci944356_egc946153,00.html

Nice info in PDFs, with kernel patch scripts;
http://www.informatik.uni-bremen.de/agbs/qualifikationsarbeiten/diplomarbeiten/2005_efkemann.pdf
http://www.ccur.com/isdmanuals/2RedHawk%20Linux/0898004-430_RedHawk_Linux_2.X_Users_Guide.pdf

0
2/1/2006 3:55:48 PM
Reply: