run a job on both processors of a bi-processor xeon

Hi everybody, I want to run a job on both processors of a bi-processor xeon (2x1.7Go). when I simply run the job like this: prompt>./myjob it runs only on one processor of 1.7Ghz. This takes a long time untill the job is done! Is there any simple bash command to do so ? or do I need an additional parallelisation software like PVM, OpenMP or MPI ? thanx

roberto wrote: > Hi everybody, > I want to run a job on both processors of a bi-processor xeon > (2x1.7Go). > when I simply run the job like this: > prompt>./myjob > it runs only on one processor of 1.7Ghz. This takes a long time untill > the job is done! > Is there any simple bash command to do so ? or do I need an additional > parallelisation software like PVM, OpenMP or MPI ? The program has to be written to take advantage of multiple processors. This would usually be done by implementing the program as threads. -- Paul Black mailto:paul.black@oxsemi.com Oxford Semiconductor Ltd http://www.oxsemi.com 25 Milton Park, Abingdon, Tel: +44 (0) 1235 824 909 Oxfordshire. OX14 4SH Fax: +44 (0) 1235 821 141

calogero_roberto@yahoo.fr (roberto), In a message on 18 Jan 2005 07:02:22 -0800, wrote : r> Hi everybody, r> I want to run a job on both processors of a bi-processor xeon r> (2x1.7Go). r> when I simply run the job like this: r> prompt>./myjob r> it runs only on one processor of 1.7Ghz. This takes a long time untill r> the job is done! r> Is there any simple bash command to do so ? or do I need an additional r> parallelisation software like PVM, OpenMP or MPI ? r> thanx As Paul Black said, you have to re-write your code (presumably C or C++) to use threads. When you do, each thread runs in parallel on a separate processor. Note this only make sense if the algorithm can be parallelized this way. Often this is possible if there are either separate non-sequential operations (eg where operation B does not need A to be complete). Often array operations are that way -- if you want to say add a constant to an array (such as an image), you can divide the array into to halves (no, you don't have to copy the data, just have two sets of indexing values): You create a data structure containing the indexing info: typedef struct { float *array; int istart, iend; float offset; } ThreadData; After including the thread headers: /* POSIX Thread header */ #include <pthread.h> #include <errno.h> #include <stdio.h> /* Thread function */ static void *thread_function(void *argument); #define NUMBEROFPROCESSORS 2 /* Change this as needed */ void AddOffsetToArrayParallel (float *array, int array_size, float offset) { static ThreadData threaddata[NUMBEROFPROCESSORS]; static pthread_t threads[NUMBEROFPROCESSORS]; static long int threadreturn; int ithread, elements_per_thread, tstart, tend; elements_per_thread = array_size / NUMBEROFPROCESSORS; tstart = 0; for (ithread = 0; ithread < NUMBEROFPROCESSORS; ithread++) { threaddata[ithread].array = array; threaddata[ithread].offset = offset; threaddata[ithread].istart = tstart; threaddata[ithread].iend = tstart+elements_per_thread; if (threaddata[ithread].iend >= array_size) { threaddata[ithread].iend = array_size-1; } tstart = threaddata[ithread].iend+1; if (pthread_create(&threads[ithread], NULL, thread_function, (void *)&threaddata[ithread]) != 0) { perror("Failed to start thread in AddOffsetToArrayParallel"); exit(errno); } } for (ithread = 0; ithread < NUMBEROFPROCESSORS; ithread++) { if (pthread_join(&threads[ithread], (void **)&threadreturn) != 0) { perror("Failed to join to thread in AddOffsetToArrayParallel"); exit(errno); } } } static void *thread_function(void *argument) { ThreadData *targs = (ThreadData *) argument; int i, is, ie; float *ar, off; static long int dummy = 0; is = targs->istart; ie = targs->iend; ar = targs->array; off = targs->offset; for (i = is; i <= ie; i++) { ar[i] += off; } return (void *) &dummy; } Alternitively, if you have two *independent* operations (eg not dependent on each other), you can use pthread_create to launch each of the functions and then use pthread_join to wait for them to complete if/when you need to execute code that is dependent on both being complete (at the very least you should call pthread_join before your program exits to clean up all of the loose ends... If your job is a script runing several *different* programs that are indepentent, it is easier -- you just fork things in pairs: #!/bin/sh program1 & program2 & wait program3 & program4 & wait r> \/ Robert Heller ||InterNet: heller@cs.umass.edu http://vis-www.cs.umass.edu/~heller || heller@deepsoft.com http://www.deepsoft.com /\FidoNet: 1:321/153

roberto wrote: > Hi everybody, > I want to run a job on both processors of a bi-processor xeon > (2x1.7Go). > when I simply run the job like this: > prompt>./myjob > it runs only on one processor of 1.7Ghz. This takes a long time untill > the job is done! > Is there any simple bash command to do so ? or do I need an additional > parallelisation software like PVM, OpenMP or MPI ? > thanx The only easy way, and that applies only in certain circumstances, is what I do with BOINC projects like setiathome. There I run 4 instances (I have two hyperthreaded Xeon processors), each one on a separate data set. So no one process goes any faster (in fact they all go somewhat slower), but the total throughput is increased. -- .~. Jean-David Beyer Registered Linux User 85642. /V\ PGP-Key: 9A2FC99A Registered Machine 241939. /( )\ Shrewsbury, New Jersey http://counter.li.org ^^-^^ 14:05:00 up 18 days, 3:25, 3 users, load average: 4.19, 4.20, 4.13

roberto wrote: > Hi everybody, > I want to run a job on both processors of a bi-processor xeon > (2x1.7Go). > when I simply run the job like this: > prompt>./myjob > it runs only on one processor of 1.7Ghz. This takes a long time untill > the job is done! > Is there any simple bash command to do so ? or do I need an additional > parallelisation software like PVM, OpenMP or MPI ? > thanx Another option you have it's (if you can do it or it's useful to you) to launch some instances of "myjob" so they run in both processors. -- Jose Maria Lopez Hernandez Director Tecnico de bgSEC jkerouac@bgsec.com bgSEC Seguridad y Consultoria de Sistemas Informaticos http://www.bgsec.com ESPA�A The only people for me are the mad ones -- the ones who are mad to live, mad to talk, mad to be saved, desirous of everything at the same time, the ones who never yawn or say a commonplace thing, but burn, burn, burn like fabulous yellow Roman candles. -- Jack Kerouac, "On the Road"

In <288ee550.0501180702.7b88b435@posting.google.com>, on 01/18/2005 at 07:02 AM, calogero_roberto@yahoo.fr (roberto) said: >I want to run a job on both processors of a bi-processor xeon You don't have to do anything special. >prompt>./myjob That looks like you're only running a single command. If it's only a single thread in a single process then it's logically impossible to run it on two processors concurrently. Try running a job that has several processes and see what happens to your CPU distribution. -- Shmuel (Seymour J.) Metz, SysProg and JOAT <http://patriot.net/~shmuel> Unsolicited bulk E-mail subject to legal action. I reserve the right to publicly post or ridicule any abusive E-mail. Reply to domain Patriot dot net user shmuel+news to contact me. Do not reply to spamtrap@library.lspace.org

Shmuel (Seymour J.) Metz wrote: > In <288ee550.0501180702.7b88b435@posting.google.com>, on 01/18/2005 > at 07:02 AM, calogero_roberto@yahoo.fr (roberto) said: > > >>I want to run a job on both processors of a bi-processor xeon > > > You don't have to do anything special. > > >>prompt>./myjob > > > That looks like you're only running a single command. If it's only a > single thread in a single process then it's logically impossible to > run it on two processors concurrently. Try running a job that has > several processes and see what happens to your CPU distribution. > I can really see that when I run IBM's DB2 dbms. Ignoring my client process, DB2 runs a lot of processes at once. Typically 4 page cleaners, 4 page fetchers, sometimes three concurrent processing agents, and a lot of others waiting to do things but using little cpu time (deadlock detector, ....). It really helps to have SCSI hard drives and 2 hyperthreaded Xeon processors. And a lotta RAM. -- .~. Jean-David Beyer Registered Linux User 85642. /V\ PGP-Key: 9A2FC99A Registered Machine 241939. /( )\ Shrewsbury, New Jersey http://counter.li.org ^^-^^ 10:00:00 up 1 day, 18:13, 3 users, load average: 3.32, 3.27, 3.25