f



Parallel Processing in IDL

What's an easy way to use multiple processors in IDL? I have a large
program but I want to start with a simple program first. Let's say I
have an array of a million integers and I have four processors. I want
to add all the elements in this array and output it. I could split the
array into four parts and give it to each processor right? Is this
possible? How would I do this? Thanks!
0
amyusuf1 (19)
10/11/2010 4:35:16 PM
comp.lang.idl-pvwave 12260 articles. 4 followers. Post Follow

11 Replies
1615 Views

Similar Articles

[PageSpeed] 48

On 10/11/10 10:35 AM, Ammar Yusuf wrote:
> What's an easy way to use multiple processors in IDL? I have a large
> program but I want to start with a simple program first. Let's say I
> have an array of a million integers and I have four processors. I want
> to add all the elements in this array and output it. I could split the
> array into four parts and give it to each processor right? Is this
> possible? How would I do this? Thanks!

IDL's thread pool (search the IDL online help for thread pool for more 
information) will do things like that automatically if you just have a 
system with multiple processors.

If you want to do more complex tasks than just vector operations then 
check out FastDL (it has an MPI interface and a task farming interface, 
depending on the level of control you need):

   http://www.txcorp.com/products/FastDL/index.php

Disclosure: I work for Tech-X.

Mike
-- 
www.michaelgalloy.com
Research Mathematician
Tech-X Corporation
0
mgalloy (665)
10/11/2010 4:41:21 PM
On Oct 11, 10:35=A0am, Ammar Yusuf <amyus...@gmail.com> wrote:
> What's an easy way to use multiple processors in IDL? I have a large
> program but I want to start with a simple program first. Let's say I
> have an array of a million integers and I have four processors. I want
> to add all the elements in this array and output it. I could split the
> array into four parts and give it to each processor right? Is this
> possible? How would I do this? Thanks!

I would look at the documentation about thread pool.

In your case
y =3D total(array)
will take care of what you want to do assuming your thread pool is set
up properly.

from the total documentation:
"This routine is written to make use of IDL's thread pool, which can
increase execution speed on systems with multiple CPUs. The values
stored in the !CPU system variable control whether IDL uses the thread
pool for a given computation. In addition, you can use the thread pool
keywords TPOOL_MAX_ELTS, TPOOL_MIN_ELTS, and TPOOL_NOTHREAD to
override the defaults established by !CPU for a single invocation of
this routine. See Thread Pool Keywords for details. "


Cheers,
Matt
0
Matt
10/11/2010 4:41:46 PM
Check the thread pool as Matt suggests. If this does not work for you
you can also check the idl_idlbridge. Haje

On Oct 11, 12:35=A0pm, Ammar Yusuf <amyus...@gmail.com> wrote:
> What's an easy way to use multiple processors in IDL? I have a large
> program but I want to start with a simple program first. Let's say I
> have an array of a million integers and I have four processors. I want
> to add all the elements in this array and output it. I could split the
> array into four parts and give it to each processor right? Is this
> possible? How would I do this? Thanks!

0
Haje
10/11/2010 5:51:27 PM
On Oct 11, 12:41=A0pm, mgalloy <mgal...@gmail.com> wrote:
> On 10/11/10 10:35 AM, Ammar Yusuf wrote:
>
> > What's an easy way to use multiple processors in IDL? I have a large
> > program but I want to start with a simple program first. Let's say I
> > have an array of a million integers and I have four processors. I want
> > to add all the elements in this array and output it. I could split the
> > array into four parts and give it to each processor right? Is this
> > possible? How would I do this? Thanks!
>
> IDL's thread pool (search the IDL online help for thread pool for more
> information) will do things like that automatically if you just have a
> system with multiple processors.
>
> If you want to do more complex tasks than just vector operations then
> check out FastDL (it has an MPI interface and a task farming interface,
> depending on the level of control you need):
>
> =A0 =A0http://www.txcorp.com/products/FastDL/index.php
>
> Disclosure: I work for Tech-X.
>
> Mike
> --www.michaelgalloy.com
> Research Mathematician
> Tech-X Corporation

Well let's say I have a for loop and inside this for loop I have
around 50-100 lines and these computations are going out to different
procedures and functions. Would I still use the IDL Thread Pool?
Thanks!
0
amyusuf1 (19)
10/11/2010 6:06:35 PM
On 10/11/10 12:06 PM, Ammar Yusuf wrote:
> Well let's say I have a for loop and inside this for loop I have
> around 50-100 lines and these computations are going out to different
> procedures and functions. Would I still use the IDL Thread Pool?
> Thanks!

The thread pool automatically detects and uses multiple CPUs for vector 
operations and particular routines (listed below). So there is no way to 
control how work is sent to the various processors. You can control a 
few parameters via the CPU routine.

Table 12.2. Thread aware mathematical routines
ACOS	ALOG10	ALOG
CONJ	COS	COSH FINITE	FLOOR	GAMMA
ASIN ERRORF GAUSSINT ROUND
ATAN EXP IMAGINARY SIN TVSCL
FINDGEN UINDGEN FLOAT
ABS CEIL EXPINT ISHFT
LNGAMMA	MATRIX_MULTIPLY	PRODUCT SINH SQRT TAN TANH VOIGT

Table 12.3. Thread aware image processing routines
BYTSCL	CONVOL	FFT

Table 12.4. Thread aware array creation routines
INTERPOLATE DCINDGEN MAKE_ARRAY DOUBLE ULONG
POLY_2D DCOMPLEXARR REPLICATE FIX ULONG64
BINDGEN INDGEN ULINDGEN
BYTARR	CINDGEN LINDGEN	L64INDGEN
UL64INDGEN

Table 12.5. Thread aware data type conversion routines
BYTE	COMPLEX	DCOMPLEX LONG	LONG64	UINT

Table 12.6. Thread aware array manipulation routines
MAX MIN REPLICATE_INPLACE TOTAL WHERE

Table 12.7. Thread aware programming and IDL control routines
BYTEORDER	LOGICAL_AND	LOGICAL_OR	LOGICAL_TRUE

Mike
-- 
www.michaelgalloy.com
Research Mathematician
Tech-X Corporation
0
mgalloy
10/11/2010 8:01:58 PM
On 11 oct, 12:35, Ammar Yusuf <amyus...@gmail.com> wrote:
> What's an easy way to use multiple processors in IDL? I have a large
> program but I want to start with a simple program first. Let's say I
> have an array of a million integers and I have four processors. I want
> to add all the elements in this array and output it. I could split the
> array into four parts and give it to each processor right? Is this
> possible? How would I do this? Thanks!

I wrote a routine to use multi-threading some time ago. I use
IDL_IDLBridge and SHMMAP and it works good for some cases. If you are
interested I can send you a copy.
It only works for functions without output keywords.

Cheers,
nata
0
nata
10/11/2010 10:13:48 PM
On 12 Okt., 00:13, nata <bernat.puigdomen...@gmail.com> wrote:
> On 11 oct, 12:35, Ammar Yusuf <amyus...@gmail.com> wrote:
>
> > What's an easy way to use multiple processors in IDL? I have a large
> > program but I want to start with a simple program first. Let's say I
> > have an array of a million integers and I have four processors. I want
> > to add all the elements in this array and output it. I could split the
> > array into four parts and give it to each processor right? Is this
> > possible? How would I do this? Thanks!
>
> I wrote a routine to use multi-threading some time ago. I use
> IDL_IDLBridge and SHMMAP and it works good for some cases. If you are
> interested I can send you a copy.
> It only works for functions without output keywords.
>
> Cheers,
> nata

Hi Nata,
please send the copy also to me :)

Cheers

CR
0
rogass (101)
10/12/2010 12:34:46 PM
On Oct 12, 8:34=A0am, chris <rog...@googlemail.com> wrote:
> On 12 Okt., 00:13, nata <bernat.puigdomen...@gmail.com> wrote:
>
>
>
> > On 11 oct, 12:35, Ammar Yusuf <amyus...@gmail.com> wrote:
>
> > > What's an easy way to use multiple processors in IDL? I have a large
> > > program but I want to start with a simple program first. Let's say I
> > > have an array of a million integers and I have four processors. I wan=
t
> > > to add all the elements in this array and output it. I could split th=
e
> > > array into four parts and give it to each processor right? Is this
> > > possible? How would I do this? Thanks!
>
> > I wrote a routine to use multi-threading some time ago. I use
> > IDL_IDLBridge and SHMMAP and it works good for some cases. If you are
> > interested I can send you a copy.
> > It only works for functions without output keywords.
>
> > Cheers,
> > nata
>
> Hi Nata,
> please send the copy also to me :)
>
> Cheers
>
> CR

I will send you a copy this evening, I have to check the versions I
have.
Have a nice day,
nata
0
10/12/2010 1:30:30 PM
On Oct 11, 6:13=A0pm, nata <bernat.puigdomen...@gmail.com> wrote:
> On 11 oct, 12:35, Ammar Yusuf <amyus...@gmail.com> wrote:
>
> > What's an easy way to use multiple processors in IDL? I have a large
> > program but I want to start with a simple program first. Let's say I
> > have an array of a million integers and I have four processors. I want
> > to add all the elements in this array and output it. I could split the
> > array into four parts and give it to each processor right? Is this
> > possible? How would I do this? Thanks!
>
> I wrote a routine to use multi-threading some time ago. I use
> IDL_IDLBridge and SHMMAP and it works good for some cases. If you are
> interested I can send you a copy.
> It only works for functions without output keywords.
>
> Cheers,
> nata

Can you send that code to me please. Thanks!
0
amyusuf1 (19)
10/13/2010 7:38:58 PM
freelance writer
0
AvisMeadows26
2/26/2011 3:18:38 AM
Il giorno marted=EC 12 ottobre 2010 00:13:48 UTC+2, nata ha scritto:
> On 11 oct, 12:35, Ammar Yusuf <amyus...@gmail.com> wrote:
> > What's an easy way to use multiple processors in IDL? I have a large
> > program but I want to start with a simple program first. Let's say I
> > have an array of a million integers and I have four processors. I want
> > to add all the elements in this array and output it. I could split the
> > array into four parts and give it to each processor right? Is this
> > possible? How would I do this? Thanks!
>=20
> I wrote a routine to use multi-threading some time ago. I use
> IDL_IDLBridge and SHMMAP and it works good for some cases. If you are
> interested I can send you a copy.
> It only works for functions without output keywords.
>=20
> Cheers,
> nata

Hi nata!
I'm challenging with IDL multithreading and I have some problems... Can I h=
ave a copy of the code you were talking about?

Cheers

MDS
0
desilvestri
5/3/2016 2:18:54 PM
Reply: