f



Re: Deleting SAS Data from a SAS DATASET #19

On Mon, 18 Aug 2008 10:26:14 -0400, SUBSCRIBE SAS-L Chandra Gadde
<ddraj2015@GMAIL.COM> wrote:

>When I used this code, I am receiving the following warning. It completely
>deleted my prod.master_date SAS Dataset. Do you know what is going on here?
>
>WARNING: This SAS global statement is not supported in PROC SQL. It has
>been ignored.

In PROC SQL, WHERE must be a clause within a statement, not a freestanding
statement. Get rid of the semicolon immediately before the word "where".

Incidentally, I think the warning is inaccurate in referring to a WHERE
statement as a global statement.

>
>
>
>On Fri, 15 Aug 2008 13:17:25 -0500, Mary <mlhoward@AVALON.NET> wrote:
>
>>One thing you might do is to add an index on the snap_dt to the dataset;
>if that's there then you should be able to delete the records in place:
>>
>>proc sql;
>>   delete from prod.master_date;
>>  where snap_dt = "&end_dt"d;
>>quit;
>>run;
>>
>>In both the ways you  are trying now you are creating new data sets rather
>than deleting records from the current data set; it would seem to me that a
>SQL delete statement would be faster than creating new datasets even if
>there isn't an index on the date.
>>
>>-Mary
>>  ----- Original Message -----
>>  From: SUBSCRIBE SAS-L Chandra Gadde
>>  To: SAS-L@LISTSERV.UGA.EDU
>>  Sent: Friday, August 15, 2008 12:14 PM
>>  Subject: Deleting SAS Data from a SAS DATASET
>>
>>
>>  Hi All
>>
>>  I have several SAS datasets that are very very big. (50GB of size). Every
>>  month, the data is being appended to these datasets. I need to deleted
>the
>>  data which is greater than 24 months. What is the best method to do this?
>>  Please help me.
>>
>>  I tried PROC SQS and DATA STEP. But these two are taking very long time.
>>
>>  Data prod.Master_data;
>>    set prod.master_date;
>>    if snap_dt = "&end_dt"d then delete;
>>  run;
>>
>>  proc sql;
>>  crete table prod.master_date from prod.master_date
>>  where snap_dt ne "&end_dt"d;
>>  quit;
0
hs
8/18/2008 3:38:07 PM
comp.soft-sys.sas 142828 articles. 3 followers. Post Follow

0 Replies
735 Views

Similar Articles

[PageSpeed] 36

Reply:

Similar Artilces:

Re: Deleting SAS Data from a SAS DATASET
One thing you might do is to add an index on the snap_dt to the dataset; = if that's there then you should be able to delete the records in place: proc sql; delete from prod.master_date; where snap_dt =3D "&end_dt"d; quit; run; In both the ways you are trying now you are creating new data sets = rather than deleting records from the current data set; it would seem to = me that a SQL delete statement would be faster than creating new = datasets even if there isn't an index on the date.=20 -Mary ----- Original Message -----=20 From: SUBSCRIBE SAS-L Chandra Gadd...

Re: Deleting SAS Data from a SAS DATASET #9
On Fri, 15 Aug 2008 16:09:46 -0500, Mary <mlhoward@AVALON.NET> wrote: >A view helps on deletes, but I wonder how it affects performance of querying the data- wouldn't storing the data in 24 different locations cause a significant slowdown in perfomance upon querying the data versus having it all in one table that is indexed? If this data is queryied a lot but only deleted once a month, the time in querying (which probably is in peak time) could be much more important than the time in deleting (which could be run when the computer is not busy, such as nights or weekends). hi, go...

Re: Deleting SAS Data from a SAS DATASET #12
--On 8/15/08 6:03 PM -0400 Chang Chung wrote: > good point! but as shown below, you can make a view of multiple datasets > having the separate indexes, and where seems to be fine with this. > interesting. ran on sas 9.1.3 sp4 on windows. > /* make a view of two datasets appended */ > proc sql feedback; > drop view work.oneTwo; > create view oneTwo as > select * from one union select * from two > order by age; > quit; One thought on this--at least in my general approach to using views--is you probably want to avoid the ORDER BY statement--especially if i...

Re: Deleting SAS Data from a SAS DATASET #2
Chandra - this isn't my expertise area - I'm sure others may have better ideas but I'll make three comments. If your datasets have a large number of variables, in your datastep method a WHERE clause will be more efficient - the IF statement causes each record to be fully read before it can be selected for deletion. A WHERE clause would only read the single variable in the case of records that meet the criteria for deletion. Data prod.Master_date; set prod.master_date; where snap_dt ne "&end_dt"d; run; You probably would be better off changing your data model t...

Re: Deleting SAS Data from a SAS DATASET #7
A view helps on deletes, but I wonder how it affects performance of = querying the data- wouldn't storing the data in 24 different locations = cause a significant slowdown in perfomance upon querying the data versus = having it all in one table that is indexed? If this data is queryied a = lot but only deleted once a month, the time in querying (which probably = is in peak time) could be much more important than the time in deleting = (which could be run when the computer is not busy, such as nights or = weekends). =20 -Mary ----- Original Message -----=20 From: ./ ADD NAME=3DData _n...

Re: Deleting SAS Data from a SAS DATASET #15
Summary: Suggestions for maintaining large data set #iw-value=1 Chandra, You are essentially making a copy of a 50GB file every month. Of course, it takes long. Consequently you either have to copy the data more efficiently or modify how the data is stored and/or used. Have you looked at system options to make this as efficient as possible? Big buffers, many buffers. Different disk handlers for input and output would also speed up the process since it is mainly IO bound. (These topics have been discussed on SAS-L, in papers, and documentation. I have not found the techniques necessary...

Re: Deleting SAS Data from a SAS DATASET #4
Summary: PROC DATASETS; AGE statement. + VIEWs This won't help you delete data from your very big data set, but you may find this example interesting. You say you append data monthly to a big data set then when big gets too big you need to clean out the old. And that takes a very long time. However if you don't physically append but use a view to append/combine you may find it easier to get rid of the unwanted old data. Consider this code. it pushes MonthlyUpdate onto the stack of 24 data sets and the 24th data set is deleted. Then all the data sets get renamed to produce a new g...

Re: Deleting SAS Data from a SAS DATASET #18
Chandra, This is an excellent suggestion from Ian! Perhaps just recreate the = data set (you would in that case want to recreate, and thus NOT use the = SQL delete statement) once every 2-3 months and delete 2-3 months worth = of data at that time; it would seem that that wouldn't take too much = longer than deleting 1 month at a time, and thus the total time would be = reduced by as much as 50%. And this would meet my favorite efficiency marker, since I've met a lot = of people who don't want to work weekends and have yet to meet a = computer who cares, "less people time!...

Re: Deleting SAS Data from a SAS DATASET #3
The disadvantages of using SQL delete are (1) NOBS= is no longer accurate, and (2) POINT= may behave unexpectedly. If those are not issues, then the delete would be faster, with or without an index, than recreating the data set (especially if the data set has indexes that would need to be rebuilt). -- Jack Hamilton jfh@alumni.stanford.org On Aug 15, 2008, at 11:17 am, Mary wrote: > One thing you might do is to add an index on the snap_dt to the > dataset; if that's there then you should be able to delete the > records in place: > > proc sql; > delete from prod.ma...

Re: Deleting SAS Data from a SAS DATASET #10
On Fri, 15 Aug 2008 17:20:13 -0500, Mary <mlhoward@AVALON.NET> wrote: >And what about the time of the database administrator who now needs to keep track of 24 indexes to each index in the previous table? I just can't see our DB2 administrators would have ever thought to organize the data this way, though we had many very large tables, like the user had, that were purged by month; it does seem like it would be a lot of "people work" to manage 24 tables plus one index instead of just one table, even if querying didn't suffer(and I do think that it could). .... hi, Th...

Re: Deleting SAS Data from a SAS DATASET #8
On 8/15/08, Mary <mlhoward@avalon.net> wrote: > A view helps on deletes, but I wonder how it affects performance of querying the data- wouldn't storing the data in 24 different locations cause a significant slowdown in perfomance upon querying the data versus having it all in one table that is indexed? If this data is queryied a lot but only deleted once a month, the time in querying (which probably is in peak time) could be much more important than the time in deleting (which could be run when the computer is not busy, such as nights or weekends). It is not the VIEW that has an...

Re: Deleting SAS Data from a SAS DATASET #14
On Fri, 15 Aug 2008 18:54:08 -0400, Dave Scocca <dave@SCOCCA.ORG> wrote: >--On 8/15/08 6:03 PM -0400 Chang Chung wrote: > >> good point! but as shown below, you can make a view of multiple datasets >> having the separate indexes, and where seems to be fine with this. >> interesting. ran on sas 9.1.3 sp4 on windows. > >> /* make a view of two datasets appended */ >> proc sql feedback; >> drop view work.oneTwo; >> create view oneTwo as >> select * from one union select * from two >> order by age; >> quit; > >One thought on this--at least in my general approach to using views--is you >probably want to avoid the ORDER BY statement--especially if it's the variable >you're indexing on and you want to use a subset. > >For any amount of data above a trivial one, there's no sense in forcing the >view to re-order th >e data if it doesn't have to. Eliminating ORDER BY is not going to do any good unless the ALL option is invoked for the UNION operator. Otherwise the view is going to purge duplicate rows, and will sort the rows to support that process. Ian mentioned the ALL option but did not emphasize its importance. > >Dave ...

Re: Deleting SAS Data from a SAS DATASET #16
Summary: SAS Scalable Performance Data Server #iw-value=1 Chandra, You now have a number of reasonable suggestions to your problem that place the burden on the programmer, but I did not see mention of the SAS Scalable Performance Data Server. This is a SAS product designed to handle large amounts of data. Considering the size of your data set suggests that perhaps it is time to spend money on the problem. Even the SPDE engine included with the base product might help, particularly if the computer has several CPUs. Ian Whitlock =============== Date: Fri, 15 Aug 2008 13:14:22 -0400...

Re: Deleting SAS Data from a SAS DATASET #13
I am not expert with indexes, but I would expect that an index on each data set would be employed even when the data are accessed through a view. If that is correct, there would be clear advantage to indexing data for each month separately. If the data are in one large file, then when old data are deleted and new data added, the entire index will need to be recreated on all of the data. That is a big operation. If, instead, data are maintained in separate files for each month and each month of data is indexed independently, then indexing needs to be performed only for the new data. The ind...

Re: Deleting SAS Data from a SAS DATASET #6
I have been watching this thread today and I just now recall an example that Paul Dorfman gave at a RUG several years ago and no, I don't recall exactly where or when other than it was NESUG or SESUG in the past 5 years. Anyway, Paul had a client who was storing something like the past 12 months of transactions as variables andat the start of each month needed to stick the just ended month on the end and drop the oldest month. Paul used Peek to read the last 11 months of each obs as a single chunk of data, append the current's month, and then used Poke to write the whole new obs as a ...

Re: Deleting SAS Data from a SAS DATASET #11
And what about the time of the database administrator who now needs to = keep track of 24 indexes to each index in the previous table? I just = can't see our DB2 administrators would have ever thought to organize the = data this way, though we had many very large tables, like the user had, = that were purged by month; it does seem like it would be a lot of = "people work" to manage 24 tables plus one index instead of just one = table, even if querying didn't suffer(and I do think that it could).=20 -Mary ----- Original Message -----=20 From: Chang Chung=20 To: SAS-L@L...

Re: Deleting SAS Data from a SAS DATASET #17
When I used this code, I am receiving the following warning. It completely deleted my prod.master_date SAS Dataset. Do you know what is going on here? WARNING: This SAS global statement is not supported in PROC SQL. It has been ignored. On Fri, 15 Aug 2008 13:17:25 -0500, Mary <mlhoward@AVALON.NET> wrote: >One thing you might do is to add an index on the snap_dt to the dataset; if that's there then you should be able to delete the records in place: > >proc sql; > delete from prod.master_date; > where snap_dt = "&end_dt"d; >quit; >run; > &...

Re: Deleting SAS Data from a SAS DATASET #5
I also read that this could be a problem on machines that have multiple = processors, such as running DB2; if the data is clustered and each = processor takes care of one cluster, then you could have queries all = hitting the same processor instead of distributing the load across all = processors, thus potentially slowing down queries in peak periods where = you have multiple queries at the same time, if queries tend to be based = on the month (such as bank statements). Thus I'd proceed with caution = on the idea of splitting one table into 24 tables with one view without = making sure th...

Deleting SAS Data from a SAS DATASET
Hi All I have several SAS datasets that are very very big. (50GB of size). Every month, the data is being appended to these datasets. I need to deleted the data which is greater than 24 months. What is the best method to do this? Please help me. I tried PROC SQS and DATA STEP. But these two are taking very long time. Data prod.Master_data; set prod.master_date; if snap_dt = "&end_dt"d then delete; run; proc sql; crete table prod.master_date from prod.master_date where snap_dt ne "&end_dt"d; quit; Hi You could use a 'modify' statement (data step). Th...

Re: What r the data types in SAS? in Base SAS , and SAS SQL
> From: Amar Mundankar > Sent: Tuesday, July 21, 2009 8:10 AM > To: sas-l@uga.edu > Subject: What r the data types in SAS? in Base SAS , and SAS SQL > > Hi all, > What are the different data types in Base SAS and SAS SQL?? character, lengths from 1 to 32,000+ numeric: lengths from 2 to 8 dates are a subtype of numeric and are identified by their date, datetime, or time formats the new proc TSPL (Table Server Processing Language) supports ANSI data types: bigint, tinyint, etc. http://support.sas.com/documentation/cdl/en/tsag/30878/HTML/default/a003 065339.htm http://s...

Re: SAS Macro to delete a SAS dataset 351242
Montura wrote: > class delete; > runInterface: method; > delete(symget('sasDatasetName')); > endmethod; > endclass; > > > Works fine if you use plain SCL. > you could also embed a Proc Datasets > > class delete; > runInterface: method; > submit continue; > proc datasets lib=work kill nolist memtype=DATSA; > > quit; > endsubmit; > endmethod; > endclass; > > > Either way, Macro is harder to read, write and understand. Except that I've ne...

Re: SAS Macro to delete a SAS dataset 131840
well, this one reminds me another thread recently discussed: http://www.listserv.uga.edu/cgi-bin/wa?A2=ind0911c&L=sas-l&F=&S=&P=4034 On Wed, 2 Dec 2009 08:50:34 -0800, SAS Techies <sastechiesblog@GMAIL.COM> wrote: >Here?s a simple SAS Macro to delete a SAS dataset?It takes the name of >the SAS dataset as the parameter? > >%macro deletedsn(dsname); > >%if %index(&dsname,'.') eq 0 %then %do; %let lib=work; %let >dsn=&dsname; %end; >%else %if %index(&dsname,'.') eq 0 %then %do; %let lib=%scan(&dsnam...

Re: SAS Macro to delete a SAS dataset 196997
> From: SAS Techies > Subject: SAS Macro to delete a SAS dataset > > Here's a simple SAS Macro to delete a SAS dataset...It takes the name of > the SAS dataset as the parameter... > > %macro deletedsn(dsname); > > %if %index(&dsname,'.') eq 0 %then %do; %let lib=work; %let > dsn=&dsname; %end; > %else %if %index(&dsname,'.') eq 0 %then %do; %let lib=%scan(&dsname, > 1,'.'); %let dsn=%scan(&dsname,1,'.'); %end; > > proc datasets lib=&lib nolist; > delete &dsn; > qui...

Re: SAS Formats
Hi Karan, SAS formats are more than SPSS value labels. In SPSS, as you may know, value labels only associate a textual label with a (numerical) value, that's all. And with (statistical) procedures both the value and its label may be printed. SAS formats can be viewed as some kind of transformation of (numerical and character) values, often intended for representation purposes (like value labels, but also quite often for conversion purposes. See the SAS docs, it's too extensive. With SAS PROCs generally only either unformatted values or formatted values are being displayed. Briefly, ...

Web resources about - Re: Deleting SAS Data from a SAS DATASET #19 - comp.soft-sys.sas

Deleting Online Predators Act of 2006 - Wikipedia, the free encyclopedia
( DOPA ) is a bill ( H.R. 5319 ) brought before the United States House of Representatives on May 9, 2006 by Republican Pennsylvania Representative ...

Are You Deleting Facebook Messages Or Unknowingly Saving Them?
You might find yourself, on occasion, the recipient or sender of hostile or just plain weird Facebook messages . And you likely delete these ...

Deleting your Facebook page is foolish
... brag, trailing closely behind that fantastically annoying “I don’t even have a TV , never mind watch the drivel!” Let’s lay it out: deleting ...

AdWords Campaign Experiments: Applying or deleting your experimental changes - YouTube
To improve your AdWords experience, we're testing a new tool that helps you optimize your account. AdWords Campaign Experiments, or ACE, does ...

Deleting abusive online content a 'near impossible' task
POSTING offensive, harassing, intimidating or defamatory content online takes mere seconds, but getting it removed can take what feels like a ...

Paleo advocate Pete Evans accused of deleting critical posts on Facebook
Is Paleo Pete Evans is open to dietary debate? Some say he's not.

Microsoft on the threshold of deleting 'appalling' Windows 8 software
MICROSOFT plans to cut its losses on Windows 8, the latest version of the software that runs the majority of the world's computers, jettisoning ...

Deleting Facebook app reportedly helps phone battery, performance and makes Facebook better
Following claims&nbsp;late last year that the iOS Facebook app was draining iPhone batteries, a new flood of similar reports is&nbsp;circling ...

Dalton McGuinty staffers broke law by deleting gas plant emails
Senior staff members in the offices of both the Ontario energy minister and former premier Dalton McGuinty intentionally deleted emails about ...

Jason Kenney under fire for praising ‘perfect, unaccented English’, deleting tweet
Canada's Minister for Multiculturalism, Jason Kenney, caused an upset Monday after praising an 11-year-old Iraqi refugee who "already speaks ...

Resources last updated: 2/23/2016 12:09:52 PM