f



SAS Parallel processing

Hi,

I am relatively new to SAS parallel processing scene and trying to
work on the following piece of code using SAS 9 on a PC. The remote
server has 4 processors and this is a UNIX based server. On my PC
windows based SAS, I execute the following code after connecting to
the remote server.

options autosignon=yes ;
rsubmit;         /* Begin simulation */
options symbolgen ls=96;
options sascmd='!sascmd -nosyntaxcheck' autosignon=yes
cpucount=ACTUAL;

%macro StatsModel(startLoop=, endLoop=, inputfn=,nsims=);
filename params "/data/&inputfn";

data parameters;
	infile params delimiter = ',' firstobs=2;
  	input runid nsims freq $ type $;
    call symput('max',_N_);
run;

%if &max < &endLoop %then %let endLoop = &max;

%let seed1 = 0; %let seed2 = -1;

%do j = &startLoop %to &endLoop;   /* Iteration for the j-th run_id */
  options sascmd='!sascmd -nosyntaxcheck' autosignon=yes;
  %syslput j = &j;
  rsubmit task&j cwait=no;

  libname dir "/data/ParallelProcess";
  %include "/sascode/Simulate.sas";
  %include "/sascode/Outputs.sas";

data parameters;
	infile params delimiter = ',' firstobs=%eval(&j +1) obs=%eval(&j+1);
  	input runid nsims freq $ type $ ;
	call symput('DIS',type);
	call symput('freqq',freq);
        label   runid = 'Simulation ID'
		freq = 'Frequency'
		type = 'Model type';
run;

proc print data = params;
 title 'Input parameters';
run;

%simulate(n=&j,freq=&freqq,DIS=&DIS);
%stats_output(n=&j,inputfn=Test.csv,startLoop=1);

endrsubmit;   /* End of remote submit for task 'j' */

%end;         /* End of 'do' loop for iteration 'j' */

rdisplay ;
waitfor _ALL_ %do i = 1 %to &endnode; jobs&i %end;;
%do i = 1 %to &endLoop;
 signoff jobs&i;
%end;

%mend StatsModel;
%StatsModel(startLoop=1,endLoop=99,nsims=1000,inputfn=Test.csv);
endrsubmit;       /* End Simulation */


I get several errors while processing the above program. I tested the
sas programs in the %include statement 'simulate.sas' and
'outputs.sas' but these do not give me errors while processed
individually. However, when I try to include the macros in these
programs into the code above, I get errors. Sometimes it also happens
that I run the program but I am not able to see the output for any of
the tasks. The log says "Remote submit to task 1 complete" (which
means the loop for j has run only for j=1, while typically I have j=1
to 10), and then the program terminates and I get no output.

Can someone help me out with this ?

Thanks a ton,
Ram.

0
prsana (10)
7/31/2007 12:51:11 AM
comp.soft-sys.sas 142828 articles. 3 followers. Post Follow

3 Replies
896 Views

Similar Articles

[PageSpeed] 22

Ram,

if (for obvious reasons since you are rsubmitting code) you can not give us
ready-to-run code then at least show us the error messages you get.

Kind regards
Robert


> -----Urspruengliche Nachricht-----
> Von Ram
> Gesendet: Dienstag, 31. Juli 2007 02:51
>
> Hi,
>
> I am relatively new to SAS parallel processing scene and trying to
> work on the following piece of code using SAS 9 on a PC. The remote
> server has 4 processors and this is a UNIX based server. On my PC
> windows based SAS, I execute the following code after connecting to
> the remote server.
>

  [ lots of code snipped / rb ]


>
>
> I get several errors while processing the above program. I tested the
> sas programs in the %include statement 'simulate.sas' and
> 'outputs.sas' but these do not give me errors while processed
> individually. However, when I try to include the macros in these
> programs into the code above, I get errors. Sometimes it also happens
> that I run the program but I am not able to see the output for any of
> the tasks. The log says "Remote submit to task 1 complete" (which
> means the loop for j has run only for j=1, while typically I have j=1
> to 10), and then the program terminates and I get no output.
>
> Can someone help me out with this ?
>
> Thanks a ton,
> Ram.
>
>
0
bardos2 (393)
7/31/2007 7:29:15 AM
Like Robert said: it is impossible to help you without knowing anything,
but your machines and how many processors they have. We don't know, where
the problem is located, if we don't know which errors you get!
One idea: you say, you test macros in your local environment? When you
call them in remote programs, you get errors? Maybe something that the
macro %xyz is unknown? That's the case if they are local to your PC and
you submit and execute code on a remote machine!

By the way: do we talk of parallel-processing, or remote-processing?

Gerhard



On Tue, 31 Jul 2007 09:29:15 +0200, Robert Bardos <bardos2@ANSYS.CH> wrote:

>Ram,
>
>if (for obvious reasons since you are rsubmitting code) you can not give
us
>ready-to-run code then at least show us the error messages you get.
>
>Kind regards
>Robert
>
>
>> -----Urspruengliche Nachricht-----
>> Von Ram
>> Gesendet: Dienstag, 31. Juli 2007 02:51
>>
>> Hi,
>>
>> I am relatively new to SAS parallel processing scene and trying to
>> work on the following piece of code using SAS 9 on a PC. The remote
>> server has 4 processors and this is a UNIX based server. On my PC
>> windows based SAS, I execute the following code after connecting to
>> the remote server.
>>
>
>  [ lots of code snipped / rb ]
>
>
>>
>>
>> I get several errors while processing the above program. I tested the
>> sas programs in the %include statement 'simulate.sas' and
>> 'outputs.sas' but these do not give me errors while processed
>> individually. However, when I try to include the macros in these
>> programs into the code above, I get errors. Sometimes it also happens
>> that I run the program but I am not able to see the output for any of
>> the tasks. The log says "Remote submit to task 1 complete" (which
>> means the loop for j has run only for j=1, while typically I have j=1
>> to 10), and then the program terminates and I get no output.
>>
>> Can someone help me out with this ?
>>
>> Thanks a ton,
>> Ram.
>>
>>
0
7/31/2007 9:52:57 AM
prsana@GMAIL.COM wrote:
>
>Hi,
>
>I am relatively new to SAS parallel processing scene and trying to
>work on the following piece of code using SAS 9 on a PC. The remote
>server has 4 processors and this is a UNIX based server. On my PC
>windows based SAS, I execute the following code after connecting to
>the remote server.
>
>options autosignon=yes ;
>rsubmit;         /* Begin simulation */
>options symbolgen ls=96;
>options sascmd='!sascmd -nosyntaxcheck' autosignon=yes
>cpucount=ACTUAL;
>
>%macro StatsModel(startLoop=, endLoop=, inputfn=,nsims=);
>filename params "/data/&inputfn";
>
>data parameters;
>         infile params delimiter = ',' firstobs=2;
>         input runid nsims freq $ type $;
>     call symput('max',_N_);
>run;
>
>%if &max < &endLoop %then %let endLoop = &max;
>
>%let seed1 = 0; %let seed2 = -1;
>
>%do j = &startLoop %to &endLoop;   /* Iteration for the j-th run_id */
>   options sascmd='!sascmd -nosyntaxcheck' autosignon=yes;
>   %syslput j = &j;
>   rsubmit task&j cwait=no;
>
>   libname dir "/data/ParallelProcess";
>   %include "/sascode/Simulate.sas";
>   %include "/sascode/Outputs.sas";
>
>data parameters;
>         infile params delimiter = ',' firstobs=%eval(&j +1)
>obs=%eval(&j+1);
>         input runid nsims freq $ type $ ;
>         call symput('DIS',type);
>         call symput('freqq',freq);
>         label   runid = 'Simulation ID'
>                 freq = 'Frequency'
>                 type = 'Model type';
>run;
>
>proc print data = params;
>  title 'Input parameters';
>run;
>
>%simulate(n=&j,freq=&freqq,DIS=&DIS);
>%stats_output(n=&j,inputfn=Test.csv,startLoop=1);
>
>endrsubmit;   /* End of remote submit for task 'j' */
>
>%end;         /* End of 'do' loop for iteration 'j' */
>
>rdisplay ;
>waitfor _ALL_ %do i = 1 %to &endnode; jobs&i %end;;
>%do i = 1 %to &endLoop;
>  signoff jobs&i;
>%end;
>
>%mend StatsModel;
>%StatsModel(startLoop=1,endLoop=99,nsims=1000,inputfn=Test.csv);
>endrsubmit;       /* End Simulation */
>
>
>I get several errors while processing the above program. I tested the
>sas programs in the %include statement 'simulate.sas' and
>'outputs.sas' but these do not give me errors while processed
>individually. However, when I try to include the macros in these
>programs into the code above, I get errors. Sometimes it also happens
>that I run the program but I am not able to see the output for any of
>the tasks. The log says "Remote submit to task 1 complete" (which
>means the loop for j has run only for j=1, while typically I have j=1
>to 10), and then the program terminates and I get no output.
>
>Can someone help me out with this ?
>
>Thanks a ton,
>Ram.

First, you need to learn how to debug macro programs.  It starts with
learning how to use options like MPRINT and MLOGIC so that you can
see what the heck your code is actualyl resolving to.

Second, I would say your underlying problem is that you are using macro
programming to do something that can be done better with other SAS
tools, in particular, SAS procs and by-processing.  Your macro code takes
so long to run that you are considering creating a parallel-processing
methodology external to SAS, which is not the way to use SAS
parallel processing tools.

I'm guessing from your code that you would benefit from reading my SGF
paper "Don't Be Loopy: Re-Sampling and Simulation the SAS(R) Way".  It
might speed up your process so much that you would be able to skip the
'parallel processing' concepts.  Here's a URL:

    http://www2.sas.com/proceedings/forum2007/183-2007.pdf

HTH,
David
--
David L. Cassell
mathematical statistician
Design Pathways
3115 NW Norwood Pl.
Corvallis OR 97330

_________________________________________________________________
A new home for Mom, no cleanup required. All starts here.
http://www.reallivemoms.com?ocid=TXT_TAGHM&loc=us
0
davidlcassell (5567)
8/15/2007 2:02:55 AM
Reply:

Similar Artilces:

Need help with SAS X command to restart to SAS Object Spawner in a SAS Stored Process.
Hi, I am trying to restart the SAS Object Spawner via a SAS Stored Process. If I test this code in Base SAS it works ok but when I use it in a Stored Process and run it in SAS Enterprise Guide or the SAS Webbapp it does not error but does not restart the Object Spawner. The .bat file contains this net stop "SAS Object Spawer" net start "SAS Object Spawner" I have enabled the x command which works ok as I can unzip a file on the O/S (Windows 2003) from a SAS STP. %let live=Yes; %if &live=Yes %THEN %DO; * Restart the SAS Object Spawner; options noxw...

Parallel Processing in SAS
I've heard that there are opportunties to do some parallel processing if your server has multiple CPUs. We have 4 CPUs running on windows 2003 SP1. Anyone have any insight or articles about Parallel processing? I heard only a few procs can actually utilize and take advantage of parallel processing, but wasn't sure. Any insight is appreciated! Jer wrote: > I've heard that there are opportunties to do some parallel processing if > your server has multiple CPUs. I think there are also opportunities for parallel processing with single CPU machines that support threading; on...

Strings: From SAS code to SAS Stored process
Hello, Your help is much appreciated. I am trying to convert my code to a stored process but it does not work smoothly. I am missing something. The line of code that starts with /*for code*/ is commented now, but if I uncomment it and erase the one below it, I can run the code from Enterprise Guide with no problem. Once I try to convert the thing to a stored process, things don't work. I use the code below as is, and I get a message saying that there is an unclosed %do statement. What is wrong??? I added the semicolon, and nothing else... Your help is much appreciated. ...

Faster SAS processing --in parallel?--
Hi - I'm rather *new* in SAS-land so please humor my lacking "SAS-code lingo" I've got a HUGE piece of SAS code that I've written for work...and I'm trying to clean it up to get it to run faster. I was *thinking*: In SQL, you can run a "SET CURRENT DEGREE=ANY" to utilize parallelism in a DB2 server and allow up to 64 instructions to be run at the same time. I typically only get 20-30 degrees of parallelism each time, but it cuts a 20-30 minute SQL query to 1 min or less... Now, with my SQL example in mind, Is there some kind of SAS command to run differ...

Re: SAS Parallel processing
Ram, if (for obvious reasons since you are rsubmitting code) you can not give us ready-to-run code then at least show us the error messages you get. Kind regards Robert > -----Urspruengliche Nachricht----- > Von Ram > Gesendet: Dienstag, 31. Juli 2007 02:51 > > Hi, > > I am relatively new to SAS parallel processing scene and trying to > work on the following piece of code using SAS 9 on a PC. The remote > server has 4 processors and this is a UNIX based server. On my PC > windows based SAS, I execute the following code after connecting to > the remote serve...

SAS parallel processing #2
I'm lucky enough to have access to SAS 9.3 on an Intel Xeon X5570 with 8 cores (at least there are 8 cpu-usage boxes in task manager). I'm using this mostly for running statistical simulations. I generate 1000 datasets and then run my macro on each of them and store the results. The macro is an alternating sequence of DATA and PROC LIFEREG steps. One thing I noticed that if I batch submit a job to process all 1000 datasets sequentially, and I open up the task manager I see the CPU usage is steady around 15%. All 8 cores have some activity, so I assume that SAS is automatically dis...

Re: Parallel Processing in SAS
Jer wrote: > I've heard that there are opportunties to do some parallel processing if > your server has multiple CPUs. I think there are also opportunities for parallel processing with single CPU machines that support threading; one tread can execute while another is waiting for I/O to complete, for example. > We have 4 CPUs running on windows 2003 SP1. > > Anyone have any insight or articles about Parallel processing? You might look through the SAS web site; there have been several SUGI papers about parallel processing, and probably some SASGloF papers as well. > I h...

SAS CONNECT/SAS Display Manager or BASE SAS
Hi, We have SAS EBI Server license. Many or our SAS Programmer wants to use BASE SAS (or SAS Display Manager) instead of SAS Enterprise Guide. But As you know Windows Server comes with only two terminal licenses. I see that we have SAS/CONNECT license (which came with BI suite). Can I install BASE SAS on PC and use SAS/CONNECT to run code on the server? Or Do we have to buy PC license for BASE SAS? What's options we have. Please help. Thanks On Oct 21, 8:11=A0am, abcu...@YAHOO.COM (Abc Unha) wrote: > Hi, > We have SAS EBI Server license. Many or our SAS Programmer wants to =A0us...

Re: SAS CONNECT/SAS Display Manager or BASE SAS
Dear SAS-L-ers, Abc Unha posted the following: > Hi, > We have SAS EBI Server license. Many or our SAS Programmer wants to > use BASE > SAS (or SAS Display Manager) instead of SAS Enterprise Guide. > > But As you know Windows Server comes with only two terminal licenses. > I see that we have SAS/CONNECT license (which came with BI suite). Can > I > install BASE SAS on PC and use SAS/CONNECT to run code on the server? > Or Do we have to buy PC license for BASE SAS? > > What's options we have. Please help. > Abc, I can't claim to know the ABC'...

What r the data types in SAS? in Base SAS , and SAS SQL
Hi all, What are the different data types in Base SAS and SAS SQL?? Regards, Amar. "Amar Mundankar" <amarmundankar@gmail.com> wrote in message news:5d3eb1f8-ea05-48f1-b41c-92b517889cbc@d4g2000yqa.googlegroups.com... > Hi all, > What are the different data types in Base SAS and SAS SQL?? > > Regards, > Amar. PROC SQL is part of base SAS, but it doesn't matter, the answer is the same wherever you are in SAS. SAS has only two data types, character and numeric. You use formats, informats, and lengths to achieve the effects other languages implement with a m...

Re: SAS Parallel processing #4
prsana@GMAIL.COM wrote: > >Hi, > >I am relatively new to SAS parallel processing scene and trying to >work on the following piece of code using SAS 9 on a PC. The remote >server has 4 processors and this is a UNIX based server. On my PC >windows based SAS, I execute the following code after connecting to >the remote server. > >options autosignon=yes ; >rsubmit; /* Begin simulation */ >options symbolgen ls=96; >options sascmd='!sascmd -nosyntaxcheck' autosignon=yes >cpucount=ACTUAL; > >%macro StatsModel(startLoop=, endLoop=, inputf...

Re: Faster SAS processing --in parallel?--
Dear SAS-L-ers, Rick posted the following interesting question: > I'm rather *new* in SAS-land so please humor my lacking > "SAS-code lingo" > > I've got a HUGE piece of SAS code that I've written for > work...and I'm trying to clean it up to get it to run faster. > I was *thinking*: > > In SQL, you can run a "SET CURRENT DEGREE=ANY" to utilize > parallelism in a DB2 server and allow up to 64 instructions > to be run at the same time. I typically only get 20-30 > degrees of parallelism each time, but it cuts a 20-30 minute ...

Re: SAS Parallel processing #3 1565310
Like Robert said: it is impossible to help you without knowing anything, but your machines and how many processors they have. We don't know, where the problem is located, if we don't know which errors you get! One idea: you say, you test macros in your local environment? When you call them in remote programs, you get errors? Maybe something that the macro %xyz is unknown? That's the case if they are local to your PC and you submit and execute code on a remote machine! By the way: do we talk of parallel-processing, or remote-processing? Gerhard On Tue, 31 Jul 2007 09:29:15 +020...

Re: Faster SAS processing --in parallel?-- #2
I've presented a couple papers at SAS Conferences on MP/CONNECT running under Version 8.2--they'll be most useful if used in conjunction with Cheryl Donninger's paper. http://www2.sas.com/proceedings/sugi26/p269-26.pdf http://www2.sas.com/proceedings/sugi27/p107-27.pdf ...

Web resources about - SAS Parallel processing - comp.soft-sys.sas

Analog signal processing - Wikipedia, the free encyclopedia
Analog signal processing is any signal processing conducted on analog signals by analog means. "Analog" indicates something that is mathematically ...

Hyatt warns of malware on its payment-processing system
Hyatt Hotels Corp. says it found malicious software on the computer system that processes customer payments, raising the possibility that hackers ...

Conexant Announces New Audio Processing Solutions At CES
At CES, Conexant gave me a demo of some of the audio processing technology that they've been working on. Conexant is a company that probably ...

Two atoms make quantum memory, processing gate, and test of entanglement
(credit: University of Maryland ) Quantum systems are inherently fragile as any interactions with the outside world can change their state. ...

Government "Processing Error" Sinks Housing Reports for Entire Year; Where to From Here?
Huge "Processing Error" in Government Housing Data When I saw some of the upwardly revised GDP estimates in 2015 I thought they were too good ...

THE PAYMENTS ECOSYSTEM: Everything you need to know about the next era of payment processing
The way we pay has changed dramatically. People are using their smartphones for every kind of formal and informal transaction — to shop at stores, ...

Audi turns to Qualcomm power for entertainment, navigation processing
Qualcomm proves its chip-making innovation, announcing a deal at CES where Audi will use its first automotive quality infotainment chipset, the ...

Google patches critical media processing and rooting vulnerabilities in Android
A new batch of security fixes for Android address flaws that could allow attackers to compromise devices via rogue emails, Web pages, and MMS ...

Feds Stop Processing NICS Denial Appeals
Obama Administration Enacts More Gun Control By Misusing FBI Background Check System

This just in: Marco Rubio's substance is thinner than an Intel processing chip
In case you hadn't already guessed it, Marco Rubio's got a smile and a stump speech, but that's about it. Reporters who have followed him from ...

Resources last updated: 2/7/2016 3:04:39 AM