Re: combine n SAS datasets in to one SAS dataset.
If you use the dictionary tables try Proc SQL - it will run much more
quickly than a datastep:
%let lib=YourLib;
%let mem=file2007;
proc sql noprint;
select MemName into :MemList
separated by " &lib.."
from dictionary.tables
where LibName=upcase("&lib")
and MemType='DATA'
and MemName eqt upcase("&mem");
quit;
data &lib.File_History;
set &lib.&MemList;
run;
hth
Paul Choate
DDS Data Extraction
(916) 654-2160
-----Original Message-----
From: SAS(r) Discussion [mailto:SAS-L@LISTSERV.UGA.EDU] On Behalf Of
stulkem@YAHOO.COM
Sent: Thursday, January 17, 2008 12:06 PM
To: SAS-L@LISTSERV.UGA.EDU
Subject: combine n SAS datasets in to one SAS dataset.
Thanks for your help in advance!
I have hundreds of SAS datasets that I want to combine into one SAS
dataset.
For example, I want to set file20070101.sas7bdat -
file20071231.sas7bdat (365 total files) into
file_history.sas7bdat.
How can I write this into a macro or array or whatever will work so I
don't have to write each individual file name??
Thanks again!
Mark
pchoate
comp.soft-sys.sas
142828 articles. 
3 followers.
0 Replies
1012 Views
Similar Artilces:
Re: combine n SAS datasets in to one SAS dataset. #4
<html> <body> <br> Filename allfiles ('d:\conv\2008*.asc'); * Filenames can use wildcards;<br><br> infile allfiles;<br><br> <br><br> <br> At 10:53 AM 1/14/2009, b.djaby@gmail.com wrote:<br> <blockquote type=cite class=cite cite="">Hi,<br> I have a similar problem in my SAS macro;<br> I have many ascii files where name are 2008011.asc to 2008123<br> year = 2008, month=01 to 12 and dekad = 1 to 3;<br> To import all my files, I have writed 2 similars macro contain with<br> those code:<br> %LET month=9;<br> %LET dekad=3;<br> <x-tab> </x-tab>%Do i= 1 %to &month;<br> <x-tab> </x-tab>%DO j = 1 %to &dekad;<br> %let dataname ="d:\conv\20080&i&j.asc";<br> infile &dataname;<br> ****<br> and for second<br><br> %LET month=12;<br> %LET dekad=3;<br> <x-tab> </x-tab>%Do i= 10 %to &month;<br> <x-tab> </x-tab>%DO j = 1 %to &dekad;...
Re: combine n SAS datasets in to one SAS dataset. #3
Here's another option using julian dates: %macro julday(year); %do i=1 %to 365; data _null_; day=datejul(&year*1000+&i); date=put(day,yymmddn8.); call symput("yymmdd",date); put day= date=; run; %put yymmdd=&yymmdd; proc append base=file_history new=file&yymmdd.;run; %end; %mend; %julday(2007); At 11:44 AM 1/18/2008, Howard Schreier <hs AT dc-sug DOT org> wrote: >On Thu, 17 Jan 2008 12:06:01 -0800, stulkem@YAHOO.COM wrote: > > >Thanks for your help in advance! > > > >I have hundreds of SAS datasets that I want to combine into one SAS > >dataset. > > > >For example, I want to set file20070101.sas7bdat - > >file20071231.sas7bdat (365 total files) into > >file_history.sas7bdat. > > > >How can I write this into a macro or array or whatever will work so I > >don't have to write each individual file name?? > > > >Thanks again! > > > >Mark > >You must provide some rule or process which SAS can use to discover the >names of the data sets to be processed. > >Here is how I might deal with the example (one file for each day of a >calendar year): > > data _null_; > call execute('proc datasets library=mylib nolist;'); > call execute('delete file_history;'); > call execute('run;'); > do day = '01jan2007'd to '31dec2007'd; > call execute('ap...
Re: combine n SAS datasets in to one SAS dataset. #2
On Thu, 17 Jan 2008 12:06:01 -0800, stulkem@YAHOO.COM wrote: >Thanks for your help in advance! > >I have hundreds of SAS datasets that I want to combine into one SAS >dataset. > >For example, I want to set file20070101.sas7bdat - >file20071231.sas7bdat (365 total files) into >file_history.sas7bdat. > >How can I write this into a macro or array or whatever will work so I >don't have to write each individual file name?? > >Thanks again! > >Mark You must provide some rule or process which SAS can use to discover the names of the data sets to be processed. Here is how I might deal with the example (one file for each day of a calendar year): data _null_; call execute('proc datasets library=mylib nolist;'); call execute('delete file_history;'); call execute('run;'); do day = '01jan2007'd to '31dec2007'd; call execute('append base=mylib.file_history data=mylib.file' || put(day,yymmddn8.) || ';'); end; call execute('quit;'); run; ...
combine n SAS datasets in to one SAS dataset.
Thanks for your help in advance! I have hundreds of SAS datasets that I want to combine into one SAS dataset. For example, I want to set file20070101.sas7bdat - file20071231.sas7bdat (365 total files) into file_history.sas7bdat. How can I write this into a macro or array or whatever will work so I don't have to write each individual file name?? Thanks again! Mark On Jan 17, 3:06=A0pm, stul...@yahoo.com wrote: > Thanks for your help in advance! > > I have hundreds of SAS datasets that I want to combine into one SAS > dataset. > > For example, =A0I want to set =A0 =A0 file20070101.sas7bdat - > file20071231.sas7bdat =A0(365 total files) into > file_history.sas7bdat. > > How can I write this into a macro or array or whatever will work so I > don't have to write each individual file name?? > > Thanks again! > > Mark Something like this may work. %macro DescribeLibrary(libref); data LibContents1; set sashelp.vtable; where upcase(libname)=3Dupcase("&libref") and memtype=3D'DATA'; run; proc sort data=3DLibContents1; by memname; run; data _null_; set LibContents1 end=3Deof; if memname=3D:'2007'; call symput('memname'||trim(left(put(_n_, 8.))),trim(left(memname))); if eof then call symput('nmembers',trim(left(put(_n_,8.)))); run; %put num_members=3D&nmembers; %do x=3D1 %to &nmembers; proc append base=3D&libref...
Transposing a multivariate sas dataset into a multivariate sas dataset
I currently run this code to generate a report in Excel format for a non-SAS user. data old; input lname $ label1 $ label2 $ label3 $ name $ var1 $ var2 $ var3 $; cards; Name label1 label2 label3 Johnny Jv1 Jv2 Jv3 ; run; DATA _NULL_; SET old; ARRAY report (4) $ name var1 var2 var3; ARRAY report2 (4) $ lname label1 label2 label3; FILE 'c:/desktop/file.xls' dlm='09'x; DO i=1 to 4; put report2(i) report(i); end; run; This is the output in the excel file: Name Johnny label1 Jv1 label2 Jv2 label3 Jv3 My report has changed such that there will be multiple records. data old; input lname $ label1 $ label2 $ label3 $ name $ var1 $ var2 $ var3 $; cards; Name label1 label2 label3 Johnny Jv1 Jv2 Jv3 Name label1 label2 label3 Susie Sv1 Sv2 Sv3 Name label1 label2 label3 Karen Kv1 Kv2 Kv3 ; run; I would like some help in creating the output below in one excel file: Name Johnny Susie Karen label1 Jv1 Sv1 Kv1 label2 Jv2 Sv2 Kv2 label3 Jv3 Sv3 Kv3 Anyone have experience doing something like this? Thank you in advance! ...
Re: SAS System Viewer 9.1 unable to open SAS datasets ( SAS 9.1.3)
A question in return: is your table from windows? compressed from another system? and specially with personnel formats hard coded inside? Andre msiddu2000 a �crit : > SAS System Viewer 9.1 was able to open datasets that were created by > V8 engine (SAS 8.2), but is unable to open datasets that are created > with V9 engine (SAS 9.1.3). > > Does this need SAS System Viewer to be upgraded ? > or > Something else needs to be done ? > > -- Andr� WIELKI INED (Institut National d'Etudes D�mographiques) Service Informatique 133 Boulevard Davout 75980 Paris Cedex 20 m�l : wielki@ined.fr t�l : 33 (0) 1 56 06 21 54 ...
Re: How to read a SAS log into a SAS dataset.
lazybone2k@GMAIL.COM wrote: > >Dear SAS-L listers, >This may be the millionth time somebody is asking this question " how to >read a SAS log into a SAS dataset?". I looked at previous posts and >gathered info from various SUGI papers and found a way to get the log and >read it. But i dont know for some reason its not working. Please help me. >I used a Proc printto procedure to print the log to a external file >location and try to read in the log and create a SAS dataset which will >contain info about Filename, Owner name, Group name, Elapsed time, CPU >t...
Re: Deleting SAS Data from a SAS DATASET
One thing you might do is to add an index on the snap_dt to the dataset; = if that's there then you should be able to delete the records in place: proc sql; delete from prod.master_date; where snap_dt =3D "&end_dt"d; quit; run; In both the ways you are trying now you are creating new data sets = rather than deleting records from the current data set; it would seem to = me that a SQL delete statement would be faster than creating new = datasets even if there isn't an index on the date.=20 -Mary ----- Original Message -----=20 From: SUBSCRIBE SAS-L Chandra Gadde=20 To: SAS-L@LISTSERV.UGA.EDU=20 Sent: Friday, August 15, 2008 12:14 PM Subject: Deleting SAS Data from a SAS DATASET Hi All I have several SAS datasets that are very very big. (50GB of size). = Every month, the data is being appended to these datasets. I need to deleted = the data which is greater than 24 months. What is the best method to do = this? Please help me. I tried PROC SQS and DATA STEP. But these two are taking very long = time. Data prod.Master_data; set prod.master_date; if snap_dt =3D "&end_dt"d then delete; run; proc sql; crete table prod.master_date from prod.master_date where snap_dt ne "&end_dt"d; quit; Hi You've got good ideas how to delete the old observations. My modify proposal was wrong. Sorry about that! I still think that changing the data model would be a good idea. Regards Patrick ...
Re: SAS
Chandu, You should be able to find some info in the SPD Engine section of the on-line documentation. HTH, Ken -----Original Message----- From: chandu.isi@GMAIL.COM To: SAS-L@LISTSERV.UGA.EDU Sent: Mon, 3 Jul 2006 09:03:00 -0700 Subject: SAS - 9; Partition of SAS Datasets. Hi All, Recently I heared from some of my friends...that SAS - 9 supports PARTITION of SAS datasets. I tried for getting syntax and some details, but couldn't able to get. Is anyone has any idea about the partitions of SAS datasets, if so can any one help me out on this? Thanks in advance. Chandu. _____...
Re: Deleting SAS Data from a SAS DATASET #8
On 8/15/08, Mary <mlhoward@avalon.net> wrote: > A view helps on deletes, but I wonder how it affects performance of querying the data- wouldn't storing the data in 24 different locations cause a significant slowdown in perfomance upon querying the data versus having it all in one table that is indexed? If this data is queryied a lot but only deleted once a month, the time in querying (which probably is in peak time) could be much more important than the time in deleting (which could be run when the computer is not busy, such as nights or weekends). It is not the VIEW that has an influence on deleting the old data. I would think that having the 24 indexed data sets might be about as good as having the giant data set. I would think the indexes could be used while accessing the data through views. Where's that guy that says he knows everything about using indexed data sets? I would agree that much depends on how the data is used. And I don't know the answers to those questions. > > -Mary > ----- Original Message ----- > From: ./ ADD NAME=Data _null_, > To: SAS-L@LISTSERV.UGA.EDU > Sent: Friday, August 15, 2008 3:51 PM > Subject: Re: Deleting SAS Data from a SAS DATASET > > > Summary: PROC DATASETS; AGE statement. + VIEWs > > This won't help you delete data from your very big data set, but you > may find this example interesting. > > You say you append data monthly to a big data set then when b...
Re: Deleting SAS Data from a SAS DATASET #4
Summary: PROC DATASETS; AGE statement. + VIEWs This won't help you delete data from your very big data set, but you may find this example interesting. You say you append data monthly to a big data set then when big gets too big you need to clean out the old. And that takes a very long time. However if you don't physically append but use a view to append/combine you may find it easier to get rid of the unwanted old data. Consider this code. it pushes MonthlyUpdate onto the stack of 24 data sets and the 24th data set is deleted. Then all the data sets get renamed to produce a new group of 24. You can see from the notes how the operation works. The data sets don't have to use a numbered range M01-M24 I did that for my convenience. proc datasets library=work; age MonthlyUpdate m01-m24; run; quit; NOTE: Deleting WORK.M24 (memtype=DATA). NOTE: Aging the name WORK.M23 to WORK.M24 (memtype=DATA). NOTE: Aging the name WORK.M22 to WORK.M23 (memtype=DATA). NOTE: Aging the name WORK.M21 to WORK.M22 (memtype=DATA). NOTE: Aging the name WORK.M20 to WORK.M21 (memtype=DATA). NOTE: Aging the name WORK.M19 to WORK.M20 (memtype=DATA). NOTE: Aging the name WORK.M18 to WORK.M19 (memtype=DATA). NOTE: Aging the name WORK.M17 to WORK.M18 (memtype=DATA). NOTE: Aging the name WORK.M16 to WORK.M17 (memtype=DATA). NOTE: Aging the name WORK.M15 to WORK.M16 (memtype=DATA). NOTE: Aging the name WORK.M14 to WORK.M15 (memtype=DATA). NOTE: Aging the name WORK.M13 to WORK.M14 (me...
Re: Deleting SAS Data from a SAS DATASET #9
On Fri, 15 Aug 2008 16:09:46 -0500, Mary <mlhoward@AVALON.NET> wrote: >A view helps on deletes, but I wonder how it affects performance of querying the data- wouldn't storing the data in 24 different locations cause a significant slowdown in perfomance upon querying the data versus having it all in one table that is indexed? If this data is queryied a lot but only deleted once a month, the time in querying (which probably is in peak time) could be much more important than the time in deleting (which could be run when the computer is not busy, such as nights or weekends). hi, good point! but as shown below, you can make a view of multiple datasets having the separate indexes, and where seems to be fine with this. interesting. ran on sas 9.1.3 sp4 on windows. cheers, chang /* test datasets */ data one two; set sashelp.class; ds = "one"; output one; ds = "two"; output two; run; /* separately index on age */ proc datasets lib=work; modify one; index delete age; index create age; modify two; index delete age; index create age; quit; /* make a view of two datasets appended */ proc sql feedback; drop view work.oneTwo; create view oneTwo as select * from one union select * from two order by age; quit; /* where is utilizing the indexes */ options msglevel=i; data sixteenAndOlder; set oneTwo; /* we are using the view */ where age >= 16; run; /* on log INFO: Index Age selected for WHERE clause optimiz...
Re: Deleting SAS Data from a SAS DATASET #19
On Mon, 18 Aug 2008 10:26:14 -0400, SUBSCRIBE SAS-L Chandra Gadde <ddraj2015@GMAIL.COM> wrote: >When I used this code, I am receiving the following warning. It completely >deleted my prod.master_date SAS Dataset. Do you know what is going on here? > >WARNING: This SAS global statement is not supported in PROC SQL. It has >been ignored. In PROC SQL, WHERE must be a clause within a statement, not a freestanding statement. Get rid of the semicolon immediately before the word "where". Incidentally, I think the warning is inaccurate in referring to a WHERE statement as a global statement. > > > >On Fri, 15 Aug 2008 13:17:25 -0500, Mary <mlhoward@AVALON.NET> wrote: > >>One thing you might do is to add an index on the snap_dt to the dataset; >if that's there then you should be able to delete the records in place: >> >>proc sql; >> delete from prod.master_date; >> where snap_dt = "&end_dt"d; >>quit; >>run; >> >>In both the ways you are trying now you are creating new data sets rather >than deleting records from the current data set; it would seem to me that a >SQL delete statement would be faster than creating new datasets even if >there isn't an index on the date. >> >>-Mary >> ----- Original Message ----- >> From: SUBSCRIBE SAS-L Chandra Gadde >> To: SAS-L@LISTSERV.UGA.EDU >> Sent: Friday, August ...
Re: Deleting SAS Data from a SAS DATASET #18
Chandra, This is an excellent suggestion from Ian! Perhaps just recreate the = data set (you would in that case want to recreate, and thus NOT use the = SQL delete statement) once every 2-3 months and delete 2-3 months worth = of data at that time; it would seem that that wouldn't take too much = longer than deleting 1 month at a time, and thus the total time would be = reduced by as much as 50%. And this would meet my favorite efficiency marker, since I've met a lot = of people who don't want to work weekends and have yet to meet a = computer who cares, "less people time!" -Mary ----- Original Message -----=20 From: Ian Whitlock=20 To: SAS-L@LISTSERV.UGA.EDU=20 Sent: Friday, August 15, 2008 6:07 PM Subject: Re: Deleting SAS Data from a SAS DATASET Summary: Suggestions for maintaining large data set #iw-value=3D1 Chandra, You are adding about 2 gig per month. That is 4% increase. Perhaps accepting the increase one month and then modifying the second month would make more sense, if the data is accessed infrequently. To do this you need to control all programs that read the data set. ...
Re: Deleting SAS Data from a SAS DATASET #10
On Fri, 15 Aug 2008 17:20:13 -0500, Mary <mlhoward@AVALON.NET> wrote: >And what about the time of the database administrator who now needs to keep track of 24 indexes to each index in the previous table? I just can't see our DB2 administrators would have ever thought to organize the data this way, though we had many very large tables, like the user had, that were purged by month; it does seem like it would be a lot of "people work" to manage 24 tables plus one index instead of just one table, even if querying didn't suffer(and I do think that it could). .... hi, There are only 24, not like thousands or millions! :-) and with any DB system, if you have a huge table, then it is more likely that it will map to multiple physical files behind the scene anyway. In terms of querying speed, i rather think it can be faster. One way is to spawn many sas sessions to do multiple separate queries at the same time, then put together... sort of like do-it-yourself parallel processing. implementation is left as homework. happy friday! cheers, chang ...
Re: Deleting SAS Data from a SAS DATASET #15
Summary: Suggestions for maintaining large data set #iw-value=1 Chandra, You are essentially making a copy of a 50GB file every month. Of course, it takes long. Consequently you either have to copy the data more efficiently or modify how the data is stored and/or used. Have you looked at system options to make this as efficient as possible? Big buffers, many buffers. Different disk handlers for input and output would also speed up the process since it is mainly IO bound. (These topics have been discussed on SAS-L, in papers, and documentation. I have not found the techniques necessary, but then I have not worked with 50 GB data sets.) You are adding about 2 gig per month. That is 4% increase. Perhaps accepting the increase one month and then modifying the second month would make more sense, if the data is accessed infrequently. To do this you need to control all programs that read the data set. It might pay to combine the previous suggestion with the MODIFY command. I suspect that SAS can skip over deleted records pretty quickly. It might pay to store the data by month and each month create a new view data prod.master / view = prod.master ; set prod.mon_x ... prod.mon_y ; run ; The process could even be automated with something like %macro prod_list ( end_dt = 1jul2008 ) ; %local i mon ; %do i = -11 %to 0 ; %let mon = %sysfunc(intnx(month,"&end_dt"d,&i)) ; %let mon = %sysfunc(putn(...
Re: Deleting SAS Data from a SAS DATASET #12
--On 8/15/08 6:03 PM -0400 Chang Chung wrote: > good point! but as shown below, you can make a view of multiple datasets > having the separate indexes, and where seems to be fine with this. > interesting. ran on sas 9.1.3 sp4 on windows. > /* make a view of two datasets appended */ > proc sql feedback; > drop view work.oneTwo; > create view oneTwo as > select * from one union select * from two > order by age; > quit; One thought on this--at least in my general approach to using views--is you probably want to avoid the ORDER BY statement--especially if it's the variable you're indexing on and you want to use a subset. For any amount of data above a trivial one, there's no sense in forcing the view to re-order th e data if it doesn't have to. Dave ...
Re: Deleting SAS Data from a SAS DATASET #2
Chandra - this isn't my expertise area - I'm sure others may have better ideas but I'll make three comments. If your datasets have a large number of variables, in your datastep method a WHERE clause will be more efficient - the IF statement causes each record to be fully read before it can be selected for deletion. A WHERE clause would only read the single variable in the case of records that meet the criteria for deletion. Data prod.Master_date; set prod.master_date; where snap_dt ne "&end_dt"d; run; You probably would be better off changing your data model though - my first thought would to be to store your data as separate yearly or monthly files and then access them with a view of the past two years or 24 months. With each month or year you would delete the oldest file and change the view one time period forward. *set up the data; data time1 time2 time3 time4; do i = 1 to 100; do t = 1 to 4; if t=1 then output time1; if t=2 then output time2; if t=3 then output time3; if t=4 then output time4; end; end; run; *three successive example views; data filet1 / view=filet1; set time1 time2; run; data filet2 / view=filet2; set time2 time3; run; data filet3 / view=filet3; set time3 time4; run; One more thought - since the data are appended - if the file is not reordered and you have the record counts from each append - then you can use firsto...
Re: Deleting SAS Data from a SAS DATASET #6
I have been watching this thread today and I just now recall an example that Paul Dorfman gave at a RUG several years ago and no, I don't recall exactly where or when other than it was NESUG or SESUG in the past 5 years. Anyway, Paul had a client who was storing something like the past 12 months of transactions as variables andat the start of each month needed to stick the just ended month on the end and drop the oldest month. Paul used Peek to read the last 11 months of each obs as a single chunk of data, append the current's month, and then used Poke to write the whole new obs as a unit. The time reduction was dramatic. It probably would not work here unless you knew a whole lot about the structure of the data set and it has a very fixed structure but I was so impressed with the solution that I like to mention it whenever it seems even remotely useful. Nat Wooding Environmental Specialist III Dominion, Environmental Biology 4111 Castlewood Rd Richmond, VA 23234 Phone:804-271-5313, Fax: 804-271-2977 CONFIDENTIALITY NOTICE: This electronic message contains information which may be legally confidential and/or privileged and does not in any case represent a firm ENERGY COMMODITY bid or offer relating thereto which binds the sender without an additional express written confirmation to that effect. The information is intended solely for the individual or entity named above and access by anyone else is unauthorized. If you are not the intended recipient, any disclos...
Re: Deleting SAS Data from a SAS DATASET #7
A view helps on deletes, but I wonder how it affects performance of = querying the data- wouldn't storing the data in 24 different locations = cause a significant slowdown in perfomance upon querying the data versus = having it all in one table that is indexed? If this data is queryied a = lot but only deleted once a month, the time in querying (which probably = is in peak time) could be much more important than the time in deleting = (which could be run when the computer is not busy, such as nights or = weekends). =20 -Mary ----- Original Message -----=20 From: ./ ADD NAME=3DData _null_,=20 To: SAS-L@LISTSERV.UGA.EDU=20 Sent: Friday, August 15, 2008 3:51 PM Subject: Re: Deleting SAS Data from a SAS DATASET Summary: PROC DATASETS; AGE statement. + VIEWs This won't help you delete data from your very big data set, but you may find this example interesting. You say you append data monthly to a big data set then when big gets too big you need to clean out the old. And that takes a very long time. However if you don't physically append but use a view to append/combine you may find it easier to get rid of the unwanted old data. Consider this code. it pushes MonthlyUpdate onto the stack of 24 data sets and the 24th data set is deleted. Then all the data sets get renamed to produce a new group of 24. You can see from the notes how the operation works. The data sets don't have to use a numbered range M01-M24 I did that...
Re: Deleting SAS Data from a SAS DATASET #3
The disadvantages of using SQL delete are (1) NOBS= is no longer accurate, and (2) POINT= may behave unexpectedly. If those are not issues, then the delete would be faster, with or without an index, than recreating the data set (especially if the data set has indexes that would need to be rebuilt). -- Jack Hamilton jfh@alumni.stanford.org On Aug 15, 2008, at 11:17 am, Mary wrote: > One thing you might do is to add an index on the snap_dt to the > dataset; if that's there then you should be able to delete the > records in place: > > proc sql; > delete from prod.master_date; > where snap_dt = "&end_dt"d; > quit; > run; > > In both the ways you are trying now you are creating new data sets > rather than deleting records from the current data set; it would > seem to me that a SQL delete statement would be faster than creating > new datasets even if there isn't an index on the date. > > -Mary > ----- Original Message ----- > From: SUBSCRIBE SAS-L Chandra Gadde > To: SAS-L@LISTSERV.UGA.EDU > Sent: Friday, August 15, 2008 12:14 PM > Subject: Deleting SAS Data from a SAS DATASET > > > Hi All > > I have several SAS datasets that are very very big. (50GB of size). > Every > month, the data is being appended to these datasets. I need to > deleted the > data which is greater than 24 months. What is the best method to do > this? > Please help me. > ...
Re: Deleting SAS Data from a SAS DATASET #11
And what about the time of the database administrator who now needs to = keep track of 24 indexes to each index in the previous table? I just = can't see our DB2 administrators would have ever thought to organize the = data this way, though we had many very large tables, like the user had, = that were purged by month; it does seem like it would be a lot of = "people work" to manage 24 tables plus one index instead of just one = table, even if querying didn't suffer(and I do think that it could).=20 -Mary ----- Original Message -----=20 From: Chang Chung=20 To: SAS-L@LISTSERV.UGA.EDU=20 Sent: Friday, August 15, 2008 5:03 PM Subject: Re: Deleting SAS Data from a SAS DATASET hi, good point! but as shown below, you can make a view of multiple = datasets having the separate indexes, and where seems to be fine with this. interesting. ran on sas 9.1.3 sp4 on windows. cheers, chang /* test datasets */ data one two; set sashelp.class; ds =3D "one"; output one; ds =3D "two"; output two; run; /* separately index on age */ proc datasets lib=3Dwork; modify one; index delete age; index create age; modify two; index delete age; index create age; quit; /* make a view of two datasets appended */ proc sql feedback; drop view work.oneTwo; create view oneTwo as select * from one union select * from two order by age; quit; /* where is utili...
Re: converting sas transport files into sas datasets
Once you have assigned an xport libname you can read it in like a normal dataset. libname out xport 'C:\Documents and Settings\kdt\Desktop\class.xpt'; *- Create xport file; data out.class; set sashelp.class; run; *- Create work file from xport file; data work.class; set out.class; run; 2008/12/17 <pramila.sami@gmail.com>: > Hello All, > > Could anyone please give me the code for converting sas transport > files into sas datasets. thanks in advance. > ...
Re: Deleting SAS Data from a SAS DATASET #16
Summary: SAS Scalable Performance Data Server #iw-value=1 Chandra, You now have a number of reasonable suggestions to your problem that place the burden on the programmer, but I did not see mention of the SAS Scalable Performance Data Server. This is a SAS product designed to handle large amounts of data. Considering the size of your data set suggests that perhaps it is time to spend money on the problem. Even the SPDE engine included with the base product might help, particularly if the computer has several CPUs. Ian Whitlock =============== Date: Fri, 15 Aug 2008 13:14:22 -0400 Reply-To: SUBSCRIBE SAS-L Chandra Gadde <ddraj2015@GMAIL.COM> Sender: "SAS(r) Discussion" From: SUBSCRIBE SAS-L Chandra Gadde <ddraj2015@GMAIL.COM> Subject: Deleting SAS Data from a SAS DATASET Hi All I have several SAS datasets that are very very big. (50GB of size). Every month, the data is being appended to these datasets. I need to deleted the data which is greater than 24 months. What is the best method to do this? Please help me. I tried PROC SQS and DATA STEP. But these two are taking very long time. Data prod.Master_data; set prod.master_date; if snap_dt = "&end_dt"d then delete; run; proc sql; crete table prod.master_date from prod.master_date where snap_dt ne "&end_dt"d; quit; ...
<html> <body> <br> Filename allfiles ('d:\conv\2008*.asc'); * Filenames can use wildcards;<br><br> infile allfiles;<br><br> <br><br> <br> At 10:53 AM 1/14/2009, b.djaby@gmail.com wrote:<br> <blockquote type=cite class=cite cite="">Hi,<br> I have a similar problem in my SAS macro;<br> I have many ascii files where name are 2008011.asc to 2008123<br> year = 2008, month=01 to 12 and dekad = 1 to 3;<br> To import all my files, I have writed 2 similars macro contain with<br> those code:<br> %LET month=9;<br> %LET dekad=3;<br> <x-tab> </x-tab>%Do i= 1 %to &month;<br> <x-tab> </x-tab>%DO j = 1 %to &dekad;<br> %let dataname ="d:\conv\20080&i&j.asc";<br> infile &dataname;<br> ****<br> and for second<br><br> %LET month=12;<br> %LET dekad=3;<br> <x-tab> </x-tab>%Do i= 10 %to &month;<br> <x-tab> </x-tab>%DO j = 1 %to &dekad;...
Re: combine n SAS datasets in to one SAS dataset. #3
Here's another option using julian dates: %macro julday(year); %do i=1 %to 365; data _null_; day=datejul(&year*1000+&i); date=put(day,yymmddn8.); call symput("yymmdd",date); put day= date=; run; %put yymmdd=&yymmdd; proc append base=file_history new=file&yymmdd.;run; %end; %mend; %julday(2007); At 11:44 AM 1/18/2008, Howard Schreier <hs AT dc-sug DOT org> wrote: >On Thu, 17 Jan 2008 12:06:01 -0800, stulkem@YAHOO.COM wrote: > > >Thanks for your help in advance! > > > >I have hundreds of SAS datasets that I want to combine into one SAS > >dataset. > > > >For example, I want to set file20070101.sas7bdat - > >file20071231.sas7bdat (365 total files) into > >file_history.sas7bdat. > > > >How can I write this into a macro or array or whatever will work so I > >don't have to write each individual file name?? > > > >Thanks again! > > > >Mark > >You must provide some rule or process which SAS can use to discover the >names of the data sets to be processed. > >Here is how I might deal with the example (one file for each day of a >calendar year): > > data _null_; > call execute('proc datasets library=mylib nolist;'); > call execute('delete file_history;'); > call execute('run;'); > do day = '01jan2007'd to '31dec2007'd; > call execute('ap...
Re: combine n SAS datasets in to one SAS dataset. #2
On Thu, 17 Jan 2008 12:06:01 -0800, stulkem@YAHOO.COM wrote: >Thanks for your help in advance! > >I have hundreds of SAS datasets that I want to combine into one SAS >dataset. > >For example, I want to set file20070101.sas7bdat - >file20071231.sas7bdat (365 total files) into >file_history.sas7bdat. > >How can I write this into a macro or array or whatever will work so I >don't have to write each individual file name?? > >Thanks again! > >Mark You must provide some rule or process which SAS can use to discover the names of the data sets to be processed. Here is how I might deal with the example (one file for each day of a calendar year): data _null_; call execute('proc datasets library=mylib nolist;'); call execute('delete file_history;'); call execute('run;'); do day = '01jan2007'd to '31dec2007'd; call execute('append base=mylib.file_history data=mylib.file' || put(day,yymmddn8.) || ';'); end; call execute('quit;'); run; ...
combine n SAS datasets in to one SAS dataset.
Thanks for your help in advance! I have hundreds of SAS datasets that I want to combine into one SAS dataset. For example, I want to set file20070101.sas7bdat - file20071231.sas7bdat (365 total files) into file_history.sas7bdat. How can I write this into a macro or array or whatever will work so I don't have to write each individual file name?? Thanks again! Mark On Jan 17, 3:06=A0pm, stul...@yahoo.com wrote: > Thanks for your help in advance! > > I have hundreds of SAS datasets that I want to combine into one SAS > dataset. > > For example, =A0I want to set =A0 =A0 file20070101.sas7bdat - > file20071231.sas7bdat =A0(365 total files) into > file_history.sas7bdat. > > How can I write this into a macro or array or whatever will work so I > don't have to write each individual file name?? > > Thanks again! > > Mark Something like this may work. %macro DescribeLibrary(libref); data LibContents1; set sashelp.vtable; where upcase(libname)=3Dupcase("&libref") and memtype=3D'DATA'; run; proc sort data=3DLibContents1; by memname; run; data _null_; set LibContents1 end=3Deof; if memname=3D:'2007'; call symput('memname'||trim(left(put(_n_, 8.))),trim(left(memname))); if eof then call symput('nmembers',trim(left(put(_n_,8.)))); run; %put num_members=3D&nmembers; %do x=3D1 %to &nmembers; proc append base=3D&libref...
Transposing a multivariate sas dataset into a multivariate sas dataset
I currently run this code to generate a report in Excel format for a non-SAS user. data old; input lname $ label1 $ label2 $ label3 $ name $ var1 $ var2 $ var3 $; cards; Name label1 label2 label3 Johnny Jv1 Jv2 Jv3 ; run; DATA _NULL_; SET old; ARRAY report (4) $ name var1 var2 var3; ARRAY report2 (4) $ lname label1 label2 label3; FILE 'c:/desktop/file.xls' dlm='09'x; DO i=1 to 4; put report2(i) report(i); end; run; This is the output in the excel file: Name Johnny label1 Jv1 label2 Jv2 label3 Jv3 My report has changed such that there will be multiple records. data old; input lname $ label1 $ label2 $ label3 $ name $ var1 $ var2 $ var3 $; cards; Name label1 label2 label3 Johnny Jv1 Jv2 Jv3 Name label1 label2 label3 Susie Sv1 Sv2 Sv3 Name label1 label2 label3 Karen Kv1 Kv2 Kv3 ; run; I would like some help in creating the output below in one excel file: Name Johnny Susie Karen label1 Jv1 Sv1 Kv1 label2 Jv2 Sv2 Kv2 label3 Jv3 Sv3 Kv3 Anyone have experience doing something like this? Thank you in advance! ...
Re: SAS System Viewer 9.1 unable to open SAS datasets ( SAS 9.1.3)
A question in return: is your table from windows? compressed from another system? and specially with personnel formats hard coded inside? Andre msiddu2000 a �crit : > SAS System Viewer 9.1 was able to open datasets that were created by > V8 engine (SAS 8.2), but is unable to open datasets that are created > with V9 engine (SAS 9.1.3). > > Does this need SAS System Viewer to be upgraded ? > or > Something else needs to be done ? > > -- Andr� WIELKI INED (Institut National d'Etudes D�mographiques) Service Informatique 133 Boulevard Davout 75980 Paris Cedex 20 m�l : wielki@ined.fr t�l : 33 (0) 1 56 06 21 54 ...
Re: How to read a SAS log into a SAS dataset.
lazybone2k@GMAIL.COM wrote: > >Dear SAS-L listers, >This may be the millionth time somebody is asking this question " how to >read a SAS log into a SAS dataset?". I looked at previous posts and >gathered info from various SUGI papers and found a way to get the log and >read it. But i dont know for some reason its not working. Please help me. >I used a Proc printto procedure to print the log to a external file >location and try to read in the log and create a SAS dataset which will >contain info about Filename, Owner name, Group name, Elapsed time, CPU >t...
Re: Deleting SAS Data from a SAS DATASET
One thing you might do is to add an index on the snap_dt to the dataset; = if that's there then you should be able to delete the records in place: proc sql; delete from prod.master_date; where snap_dt =3D "&end_dt"d; quit; run; In both the ways you are trying now you are creating new data sets = rather than deleting records from the current data set; it would seem to = me that a SQL delete statement would be faster than creating new = datasets even if there isn't an index on the date.=20 -Mary ----- Original Message -----=20 From: SUBSCRIBE SAS-L Chandra Gadde=20 To: SAS-L@LISTSERV.UGA.EDU=20 Sent: Friday, August 15, 2008 12:14 PM Subject: Deleting SAS Data from a SAS DATASET Hi All I have several SAS datasets that are very very big. (50GB of size). = Every month, the data is being appended to these datasets. I need to deleted = the data which is greater than 24 months. What is the best method to do = this? Please help me. I tried PROC SQS and DATA STEP. But these two are taking very long = time. Data prod.Master_data; set prod.master_date; if snap_dt =3D "&end_dt"d then delete; run; proc sql; crete table prod.master_date from prod.master_date where snap_dt ne "&end_dt"d; quit; Hi You've got good ideas how to delete the old observations. My modify proposal was wrong. Sorry about that! I still think that changing the data model would be a good idea. Regards Patrick ...
Re: SAS
Chandu, You should be able to find some info in the SPD Engine section of the on-line documentation. HTH, Ken -----Original Message----- From: chandu.isi@GMAIL.COM To: SAS-L@LISTSERV.UGA.EDU Sent: Mon, 3 Jul 2006 09:03:00 -0700 Subject: SAS - 9; Partition of SAS Datasets. Hi All, Recently I heared from some of my friends...that SAS - 9 supports PARTITION of SAS datasets. I tried for getting syntax and some details, but couldn't able to get. Is anyone has any idea about the partitions of SAS datasets, if so can any one help me out on this? Thanks in advance. Chandu. _____...
Re: Deleting SAS Data from a SAS DATASET #8
On 8/15/08, Mary <mlhoward@avalon.net> wrote: > A view helps on deletes, but I wonder how it affects performance of querying the data- wouldn't storing the data in 24 different locations cause a significant slowdown in perfomance upon querying the data versus having it all in one table that is indexed? If this data is queryied a lot but only deleted once a month, the time in querying (which probably is in peak time) could be much more important than the time in deleting (which could be run when the computer is not busy, such as nights or weekends). It is not the VIEW that has an influence on deleting the old data. I would think that having the 24 indexed data sets might be about as good as having the giant data set. I would think the indexes could be used while accessing the data through views. Where's that guy that says he knows everything about using indexed data sets? I would agree that much depends on how the data is used. And I don't know the answers to those questions. > > -Mary > ----- Original Message ----- > From: ./ ADD NAME=Data _null_, > To: SAS-L@LISTSERV.UGA.EDU > Sent: Friday, August 15, 2008 3:51 PM > Subject: Re: Deleting SAS Data from a SAS DATASET > > > Summary: PROC DATASETS; AGE statement. + VIEWs > > This won't help you delete data from your very big data set, but you > may find this example interesting. > > You say you append data monthly to a big data set then when b...
Re: Deleting SAS Data from a SAS DATASET #4
Summary: PROC DATASETS; AGE statement. + VIEWs This won't help you delete data from your very big data set, but you may find this example interesting. You say you append data monthly to a big data set then when big gets too big you need to clean out the old. And that takes a very long time. However if you don't physically append but use a view to append/combine you may find it easier to get rid of the unwanted old data. Consider this code. it pushes MonthlyUpdate onto the stack of 24 data sets and the 24th data set is deleted. Then all the data sets get renamed to produce a new group of 24. You can see from the notes how the operation works. The data sets don't have to use a numbered range M01-M24 I did that for my convenience. proc datasets library=work; age MonthlyUpdate m01-m24; run; quit; NOTE: Deleting WORK.M24 (memtype=DATA). NOTE: Aging the name WORK.M23 to WORK.M24 (memtype=DATA). NOTE: Aging the name WORK.M22 to WORK.M23 (memtype=DATA). NOTE: Aging the name WORK.M21 to WORK.M22 (memtype=DATA). NOTE: Aging the name WORK.M20 to WORK.M21 (memtype=DATA). NOTE: Aging the name WORK.M19 to WORK.M20 (memtype=DATA). NOTE: Aging the name WORK.M18 to WORK.M19 (memtype=DATA). NOTE: Aging the name WORK.M17 to WORK.M18 (memtype=DATA). NOTE: Aging the name WORK.M16 to WORK.M17 (memtype=DATA). NOTE: Aging the name WORK.M15 to WORK.M16 (memtype=DATA). NOTE: Aging the name WORK.M14 to WORK.M15 (memtype=DATA). NOTE: Aging the name WORK.M13 to WORK.M14 (me...
Re: Deleting SAS Data from a SAS DATASET #9
On Fri, 15 Aug 2008 16:09:46 -0500, Mary <mlhoward@AVALON.NET> wrote: >A view helps on deletes, but I wonder how it affects performance of querying the data- wouldn't storing the data in 24 different locations cause a significant slowdown in perfomance upon querying the data versus having it all in one table that is indexed? If this data is queryied a lot but only deleted once a month, the time in querying (which probably is in peak time) could be much more important than the time in deleting (which could be run when the computer is not busy, such as nights or weekends). hi, good point! but as shown below, you can make a view of multiple datasets having the separate indexes, and where seems to be fine with this. interesting. ran on sas 9.1.3 sp4 on windows. cheers, chang /* test datasets */ data one two; set sashelp.class; ds = "one"; output one; ds = "two"; output two; run; /* separately index on age */ proc datasets lib=work; modify one; index delete age; index create age; modify two; index delete age; index create age; quit; /* make a view of two datasets appended */ proc sql feedback; drop view work.oneTwo; create view oneTwo as select * from one union select * from two order by age; quit; /* where is utilizing the indexes */ options msglevel=i; data sixteenAndOlder; set oneTwo; /* we are using the view */ where age >= 16; run; /* on log INFO: Index Age selected for WHERE clause optimiz...
Re: Deleting SAS Data from a SAS DATASET #19
On Mon, 18 Aug 2008 10:26:14 -0400, SUBSCRIBE SAS-L Chandra Gadde <ddraj2015@GMAIL.COM> wrote: >When I used this code, I am receiving the following warning. It completely >deleted my prod.master_date SAS Dataset. Do you know what is going on here? > >WARNING: This SAS global statement is not supported in PROC SQL. It has >been ignored. In PROC SQL, WHERE must be a clause within a statement, not a freestanding statement. Get rid of the semicolon immediately before the word "where". Incidentally, I think the warning is inaccurate in referring to a WHERE statement as a global statement. > > > >On Fri, 15 Aug 2008 13:17:25 -0500, Mary <mlhoward@AVALON.NET> wrote: > >>One thing you might do is to add an index on the snap_dt to the dataset; >if that's there then you should be able to delete the records in place: >> >>proc sql; >> delete from prod.master_date; >> where snap_dt = "&end_dt"d; >>quit; >>run; >> >>In both the ways you are trying now you are creating new data sets rather >than deleting records from the current data set; it would seem to me that a >SQL delete statement would be faster than creating new datasets even if >there isn't an index on the date. >> >>-Mary >> ----- Original Message ----- >> From: SUBSCRIBE SAS-L Chandra Gadde >> To: SAS-L@LISTSERV.UGA.EDU >> Sent: Friday, August ...
Re: Deleting SAS Data from a SAS DATASET #18
Chandra, This is an excellent suggestion from Ian! Perhaps just recreate the = data set (you would in that case want to recreate, and thus NOT use the = SQL delete statement) once every 2-3 months and delete 2-3 months worth = of data at that time; it would seem that that wouldn't take too much = longer than deleting 1 month at a time, and thus the total time would be = reduced by as much as 50%. And this would meet my favorite efficiency marker, since I've met a lot = of people who don't want to work weekends and have yet to meet a = computer who cares, "less people time!" -Mary ----- Original Message -----=20 From: Ian Whitlock=20 To: SAS-L@LISTSERV.UGA.EDU=20 Sent: Friday, August 15, 2008 6:07 PM Subject: Re: Deleting SAS Data from a SAS DATASET Summary: Suggestions for maintaining large data set #iw-value=3D1 Chandra, You are adding about 2 gig per month. That is 4% increase. Perhaps accepting the increase one month and then modifying the second month would make more sense, if the data is accessed infrequently. To do this you need to control all programs that read the data set. ...
Re: Deleting SAS Data from a SAS DATASET #10
On Fri, 15 Aug 2008 17:20:13 -0500, Mary <mlhoward@AVALON.NET> wrote: >And what about the time of the database administrator who now needs to keep track of 24 indexes to each index in the previous table? I just can't see our DB2 administrators would have ever thought to organize the data this way, though we had many very large tables, like the user had, that were purged by month; it does seem like it would be a lot of "people work" to manage 24 tables plus one index instead of just one table, even if querying didn't suffer(and I do think that it could). .... hi, There are only 24, not like thousands or millions! :-) and with any DB system, if you have a huge table, then it is more likely that it will map to multiple physical files behind the scene anyway. In terms of querying speed, i rather think it can be faster. One way is to spawn many sas sessions to do multiple separate queries at the same time, then put together... sort of like do-it-yourself parallel processing. implementation is left as homework. happy friday! cheers, chang ...
Re: Deleting SAS Data from a SAS DATASET #15
Summary: Suggestions for maintaining large data set #iw-value=1 Chandra, You are essentially making a copy of a 50GB file every month. Of course, it takes long. Consequently you either have to copy the data more efficiently or modify how the data is stored and/or used. Have you looked at system options to make this as efficient as possible? Big buffers, many buffers. Different disk handlers for input and output would also speed up the process since it is mainly IO bound. (These topics have been discussed on SAS-L, in papers, and documentation. I have not found the techniques necessary, but then I have not worked with 50 GB data sets.) You are adding about 2 gig per month. That is 4% increase. Perhaps accepting the increase one month and then modifying the second month would make more sense, if the data is accessed infrequently. To do this you need to control all programs that read the data set. It might pay to combine the previous suggestion with the MODIFY command. I suspect that SAS can skip over deleted records pretty quickly. It might pay to store the data by month and each month create a new view data prod.master / view = prod.master ; set prod.mon_x ... prod.mon_y ; run ; The process could even be automated with something like %macro prod_list ( end_dt = 1jul2008 ) ; %local i mon ; %do i = -11 %to 0 ; %let mon = %sysfunc(intnx(month,"&end_dt"d,&i)) ; %let mon = %sysfunc(putn(...
Re: Deleting SAS Data from a SAS DATASET #12
--On 8/15/08 6:03 PM -0400 Chang Chung wrote: > good point! but as shown below, you can make a view of multiple datasets > having the separate indexes, and where seems to be fine with this. > interesting. ran on sas 9.1.3 sp4 on windows. > /* make a view of two datasets appended */ > proc sql feedback; > drop view work.oneTwo; > create view oneTwo as > select * from one union select * from two > order by age; > quit; One thought on this--at least in my general approach to using views--is you probably want to avoid the ORDER BY statement--especially if it's the variable you're indexing on and you want to use a subset. For any amount of data above a trivial one, there's no sense in forcing the view to re-order th e data if it doesn't have to. Dave ...
Re: Deleting SAS Data from a SAS DATASET #2
Chandra - this isn't my expertise area - I'm sure others may have better ideas but I'll make three comments. If your datasets have a large number of variables, in your datastep method a WHERE clause will be more efficient - the IF statement causes each record to be fully read before it can be selected for deletion. A WHERE clause would only read the single variable in the case of records that meet the criteria for deletion. Data prod.Master_date; set prod.master_date; where snap_dt ne "&end_dt"d; run; You probably would be better off changing your data model though - my first thought would to be to store your data as separate yearly or monthly files and then access them with a view of the past two years or 24 months. With each month or year you would delete the oldest file and change the view one time period forward. *set up the data; data time1 time2 time3 time4; do i = 1 to 100; do t = 1 to 4; if t=1 then output time1; if t=2 then output time2; if t=3 then output time3; if t=4 then output time4; end; end; run; *three successive example views; data filet1 / view=filet1; set time1 time2; run; data filet2 / view=filet2; set time2 time3; run; data filet3 / view=filet3; set time3 time4; run; One more thought - since the data are appended - if the file is not reordered and you have the record counts from each append - then you can use firsto...
Re: Deleting SAS Data from a SAS DATASET #6
I have been watching this thread today and I just now recall an example that Paul Dorfman gave at a RUG several years ago and no, I don't recall exactly where or when other than it was NESUG or SESUG in the past 5 years. Anyway, Paul had a client who was storing something like the past 12 months of transactions as variables andat the start of each month needed to stick the just ended month on the end and drop the oldest month. Paul used Peek to read the last 11 months of each obs as a single chunk of data, append the current's month, and then used Poke to write the whole new obs as a unit. The time reduction was dramatic. It probably would not work here unless you knew a whole lot about the structure of the data set and it has a very fixed structure but I was so impressed with the solution that I like to mention it whenever it seems even remotely useful. Nat Wooding Environmental Specialist III Dominion, Environmental Biology 4111 Castlewood Rd Richmond, VA 23234 Phone:804-271-5313, Fax: 804-271-2977 CONFIDENTIALITY NOTICE: This electronic message contains information which may be legally confidential and/or privileged and does not in any case represent a firm ENERGY COMMODITY bid or offer relating thereto which binds the sender without an additional express written confirmation to that effect. The information is intended solely for the individual or entity named above and access by anyone else is unauthorized. If you are not the intended recipient, any disclos...
Re: Deleting SAS Data from a SAS DATASET #7
A view helps on deletes, but I wonder how it affects performance of = querying the data- wouldn't storing the data in 24 different locations = cause a significant slowdown in perfomance upon querying the data versus = having it all in one table that is indexed? If this data is queryied a = lot but only deleted once a month, the time in querying (which probably = is in peak time) could be much more important than the time in deleting = (which could be run when the computer is not busy, such as nights or = weekends). =20 -Mary ----- Original Message -----=20 From: ./ ADD NAME=3DData _null_,=20 To: SAS-L@LISTSERV.UGA.EDU=20 Sent: Friday, August 15, 2008 3:51 PM Subject: Re: Deleting SAS Data from a SAS DATASET Summary: PROC DATASETS; AGE statement. + VIEWs This won't help you delete data from your very big data set, but you may find this example interesting. You say you append data monthly to a big data set then when big gets too big you need to clean out the old. And that takes a very long time. However if you don't physically append but use a view to append/combine you may find it easier to get rid of the unwanted old data. Consider this code. it pushes MonthlyUpdate onto the stack of 24 data sets and the 24th data set is deleted. Then all the data sets get renamed to produce a new group of 24. You can see from the notes how the operation works. The data sets don't have to use a numbered range M01-M24 I did that...
Re: Deleting SAS Data from a SAS DATASET #3
The disadvantages of using SQL delete are (1) NOBS= is no longer accurate, and (2) POINT= may behave unexpectedly. If those are not issues, then the delete would be faster, with or without an index, than recreating the data set (especially if the data set has indexes that would need to be rebuilt). -- Jack Hamilton jfh@alumni.stanford.org On Aug 15, 2008, at 11:17 am, Mary wrote: > One thing you might do is to add an index on the snap_dt to the > dataset; if that's there then you should be able to delete the > records in place: > > proc sql; > delete from prod.master_date; > where snap_dt = "&end_dt"d; > quit; > run; > > In both the ways you are trying now you are creating new data sets > rather than deleting records from the current data set; it would > seem to me that a SQL delete statement would be faster than creating > new datasets even if there isn't an index on the date. > > -Mary > ----- Original Message ----- > From: SUBSCRIBE SAS-L Chandra Gadde > To: SAS-L@LISTSERV.UGA.EDU > Sent: Friday, August 15, 2008 12:14 PM > Subject: Deleting SAS Data from a SAS DATASET > > > Hi All > > I have several SAS datasets that are very very big. (50GB of size). > Every > month, the data is being appended to these datasets. I need to > deleted the > data which is greater than 24 months. What is the best method to do > this? > Please help me. > ...
Re: Deleting SAS Data from a SAS DATASET #11
And what about the time of the database administrator who now needs to = keep track of 24 indexes to each index in the previous table? I just = can't see our DB2 administrators would have ever thought to organize the = data this way, though we had many very large tables, like the user had, = that were purged by month; it does seem like it would be a lot of = "people work" to manage 24 tables plus one index instead of just one = table, even if querying didn't suffer(and I do think that it could).=20 -Mary ----- Original Message -----=20 From: Chang Chung=20 To: SAS-L@LISTSERV.UGA.EDU=20 Sent: Friday, August 15, 2008 5:03 PM Subject: Re: Deleting SAS Data from a SAS DATASET hi, good point! but as shown below, you can make a view of multiple = datasets having the separate indexes, and where seems to be fine with this. interesting. ran on sas 9.1.3 sp4 on windows. cheers, chang /* test datasets */ data one two; set sashelp.class; ds =3D "one"; output one; ds =3D "two"; output two; run; /* separately index on age */ proc datasets lib=3Dwork; modify one; index delete age; index create age; modify two; index delete age; index create age; quit; /* make a view of two datasets appended */ proc sql feedback; drop view work.oneTwo; create view oneTwo as select * from one union select * from two order by age; quit; /* where is utili...
Re: converting sas transport files into sas datasets
Once you have assigned an xport libname you can read it in like a normal dataset. libname out xport 'C:\Documents and Settings\kdt\Desktop\class.xpt'; *- Create xport file; data out.class; set sashelp.class; run; *- Create work file from xport file; data work.class; set out.class; run; 2008/12/17 <pramila.sami@gmail.com>: > Hello All, > > Could anyone please give me the code for converting sas transport > files into sas datasets. thanks in advance. > ...
Re: Deleting SAS Data from a SAS DATASET #16
Summary: SAS Scalable Performance Data Server #iw-value=1 Chandra, You now have a number of reasonable suggestions to your problem that place the burden on the programmer, but I did not see mention of the SAS Scalable Performance Data Server. This is a SAS product designed to handle large amounts of data. Considering the size of your data set suggests that perhaps it is time to spend money on the problem. Even the SPDE engine included with the base product might help, particularly if the computer has several CPUs. Ian Whitlock =============== Date: Fri, 15 Aug 2008 13:14:22 -0400 Reply-To: SUBSCRIBE SAS-L Chandra Gadde <ddraj2015@GMAIL.COM> Sender: "SAS(r) Discussion" From: SUBSCRIBE SAS-L Chandra Gadde <ddraj2015@GMAIL.COM> Subject: Deleting SAS Data from a SAS DATASET Hi All I have several SAS datasets that are very very big. (50GB of size). Every month, the data is being appended to these datasets. I need to deleted the data which is greater than 24 months. What is the best method to do this? Please help me. I tried PROC SQS and DATA STEP. But these two are taking very long time. Data prod.Master_data; set prod.master_date; if snap_dt = "&end_dt"d then delete; run; proc sql; crete table prod.master_date from prod.master_date where snap_dt ne "&end_dt"d; quit; ...
Web resources about - Re: combine n SAS datasets in to one SAS dataset. - comp.soft-sys.sas Data set - Wikipedia, the free encyclopedia
For IBM mainframe term for a file, see Data set (IBM mainframe) . For the telecommunications interface device, see Modem . A dataset (or data ...
For IBM mainframe term for a file, see Data set (IBM mainframe) . For the telecommunications interface device, see Modem . A dataset (or data ...
Data : Where can I get large datasets open to the public?Answer (1 of 91): Here's the ones I've found most useful: CrunchBase, US Census, Google Public Data, Infochimps, Timetric, Factual, Freebase, ...
YouTube - Hans Rosling: Let my dataset change your mindsetHochgeladen am 31.08.2009 http://www.ted.com Talking at the US State Department this summer, Hans Rosling uses his fascinating data-bubble software ...
The Mindset, Skillset, Dataset Approach to Social Media... Social media is comprised primarily of unstructured data, which is difficult to analyze. A new approach called Mindset, Skillset, Dataset can ...
Sending large datasets to Amazon? Use the Post OfficeAmazon has introduced a new method to move large amounts of data to and from …
The Full Dataset on What VCs are Thinking About Funding in 2016... challenging and many other startups were cutting costs (as many of ours has done in late 2015). We promised that we would clean up the dataset ...
Yahoo Releases the Largest-ever Machine Learning Dataset for Researchers
Yahoo is announcing the public release of the largest-ever machine learning dataset to the research community.
Yahoo is announcing the public release of the largest-ever machine learning dataset to the research community.
IDG Contributor Network: Are datasets truly anonymized? Two well-suited researchers are going to find ...With little fanfare or formality, Adam Smith , associate professor of computer science and engineering in Penn State's School of Electrical Engineering ...
House GOP looks to shed light on EPA secret datasets
House Republicans have introduced a bill that would block the Environmental Protection Agency from crafting regulations based on “secret” scientific ...
House Republicans have introduced a bill that would block the Environmental Protection Agency from crafting regulations based on “secret” scientific ...
Guest Contribution: “Capital Control Measures: A New Dataset”Today we are pleased to present a guest contribution written by Andrés Fernández (IDB), Michael W. Klein (Tufts), Alessandro Rebucci (Johns Hopkins ...
Resources last updated: 3/17/2016 1:33:03 PM