Can proc sql read raw data from a big text file?

  • Permalink
  • submit to reddit
  • Email
  • Follow


Because I have a SQL codes provided for SQL server to read such data. 
I couldn't find any example that proc sql can read raw data from a
file.
Thank you for your answer.
Andrew
0
Reply andrewyupeng (2) 8/30/2004 1:31:48 AM

See related articles to this posting


Andrew,

No, it cannot. The only "proc" capable of reading raw data from a text file
(regardless of size) is the Data step. The only exceptions are raw text
files with a predetermined structure, such as CSV files.

Kind regards,
----------------
Paul M. Dorfman
Jacksonville, FL
----------------

> -----Original Message-----
> From: SAS(r) Discussion [mailto:SAS-L@LISTSERV.UGA.EDU] On
> Behalf Of Andrew Yu
> Sent: Sunday, August 29, 2004 9:32 PM
> To: SAS-L@LISTSERV.UGA.EDU
> Subject: Can proc sql read raw data from a big text file?
>
> Because I have a SQL codes provided for SQL server to read such data.
> I couldn't find any example that proc sql can read raw data
> from a file.
> Thank you for your answer.
> Andrew
>
0
Reply sashole (972) 8/30/2004 5:14:27 AM

Andrew:
Paul has answered your question. I would add that you can use a Data step
view to stream data through a SAS SQL WHERE clause. In a quick sketch,
data tempVW/view=tempVW;
  input v1....
run;
proc sql;
  create table tempFiltered as
  select v1,v2,.... from tempVW
  where .....
  ;
quit;

I see the Data step INPUT statement as a very powerful preprocessor for SAS
SQL. The INPUT methods for system files span the major platforms. The
SAS/ACCESS engines for RDBMS's combine with the INPUT features to make SAS
SQL far more versatile than other SQL dialects.
Sig
-----Original Message-----
From: SAS(r) Discussion [mailto:SAS-L@LISTSERV.UGA.EDU] On Behalf Of Andrew
Yu
Sent: Sunday, August 29, 2004 9:32 PM
To: SAS-L@LISTSERV.UGA.EDU
Subject: Can proc sql read raw data from a big text file?


Because I have a SQL codes provided for SQL server to read such data. I
couldn't find any example that proc sql can read raw data from a file. Thank
you for your answer. Andrew
0
Reply HERMANS1 (2684) 8/30/2004 2:31:21 PM

Could you perhaps construct some macro statements in the from statement to
achieve the same result of a separate datastep to read the data? You can of
course use the *connection to* syntax although that is only for data
already in another DBMS, ie not necessarily raw text,

HTH.

On Mon, 30 Aug 2004 10:31:21 -0400, Sigurd Hermansen <HERMANS1@WESTAT.COM>
wrote:

>Andrew:
>Paul has answered your question. I would add that you can use a Data step
>view to stream data through a SAS SQL WHERE clause. In a quick sketch,
>data tempVW/view=tempVW;
>  input v1....
>run;
>proc sql;
>  create table tempFiltered as
>  select v1,v2,.... from tempVW
>  where .....
>  ;
>quit;
>
>I see the Data step INPUT statement as a very powerful preprocessor for SAS
>SQL. The INPUT methods for system files span the major platforms. The
>SAS/ACCESS engines for RDBMS's combine with the INPUT features to make SAS
>SQL far more versatile than other SQL dialects.
>Sig
>-----Original Message-----
>From: SAS(r) Discussion [mailto:SAS-L@LISTSERV.UGA.EDU] On Behalf Of Andrew
>Yu
>Sent: Sunday, August 29, 2004 9:32 PM
>To: SAS-L@LISTSERV.UGA.EDU
>Subject: Can proc sql read raw data from a big text file?
>
>
>Because I have a SQL codes provided for SQL server to read such data. I
>couldn't find any example that proc sql can read raw data from a file.
Thank
>you for your answer. Andrew
0
Reply ben.powell1 (971) 8/31/2004 9:14:06 AM
comp.soft-sys.sas 131467 articles. 29 followers. Post

3 Replies
325 Views

Similar Articles

[PageSpeed] 55


  • Permalink
  • submit to reddit
  • Email
  • Follow


Reply:

Similar Artilces:

Re: Can proc sql read raw data from a big text file?
Andrew: Paul has answered your question. I would add that you can use a Data step view to stream data through a SAS SQL WHERE clause. In a quick sketch, data tempVW/view=tempVW; input v1.... run; proc sql; create table tempFiltered as select v1,v2,.... from tempVW where ..... ; quit; I see the Data step INPUT statement as a very powerful preprocessor for SAS SQL. The INPUT methods for system files span the major platforms. The SAS/ACCESS engines for RDBMS's combine with the INPUT features to make SAS SQL far more versatile than other SQL dialects. Sig -----Original Message----- ...

Re: Can proc sql read raw data from a big text file? #2
Could you perhaps construct some macro statements in the from statement to achieve the same result of a separate datastep to read the data? You can of course use the *connection to* syntax although that is only for data already in another DBMS, ie not necessarily raw text, HTH. On Mon, 30 Aug 2004 10:31:21 -0400, Sigurd Hermansen <HERMANS1@WESTAT.COM> wrote: >Andrew: >Paul has answered your question. I would add that you can use a Data step >view to stream data through a SAS SQL WHERE clause. In a quick sketch, >data tempVW/view=tempVW; > input v1.... >run; >proc...

Reading a text file from VB and save the data to SQL Database
The Script below is working fine. Can anybody convert these script so that it will work from VB Form with a Command Button? Script: Author: Vasu Dim OBJdbConnection Set OBJdbConnection = CreateObject("ADODB.Connection") Set RS = CreateObject ("ADODB.Recordset") Set SQLStmt = CreateObject("ADODB.Command") OBJdbConnection.ConnectionString = "dsn=ScheduleDSN;uid=sa;pwd=;DATABASE=Schedule;APP=ASP Script" OBJdbConnection.Open set rs=createobject("adodb.recordset") RS.open "cds_Cat",OBJdbConnec...

How can I Read/Write multiple sequential Binary/Text data files
Dear there, We have an x-ray CT system. The acquisition computer acquires x-ray projections and outputs multiple data files in binary format (2-byte unsigned integer) such as projection0.raw, projection1.raw, projection2.raw ... up to projection500.raw. Each file is 2*1024*768-byte big. I would like to read those files and convert to ascii files in %5.0f/n format as projection0.data ... projection500.data so that our visualization software can undersatnd the projection images. I was trying to do this conversion using Python. However, I had troubles declaring the file names using the do-loop ...

How can I read text file including "inf" to numeric array without data loss
I'd like to read text file including inf to numeric array and then plot the graph. But I can't read the inf data please help me.. I'm Labview 6.1 version user.. Thank you so much in advance.. The Frac/Exp String to Number function and the Spreadsheet to Array function both accept "inf" and return numeric (Inf). If the Read Spreadsheet FIle VI does not work, read the file as characters and do the conversion later. Lynn ...

why fopen( ) can't open a big data file? (single file, as big as 29G)
I have a data file, the size is 29G (single file). I use "fopen()" to open it, but it always fail, the returned file pointer is NULL. who knows why? thx! guru.slt@gmail.com wrote: > I have a data file, the size is 29G (single file). > > I use "fopen()" to open it, but it always fail, the returned file > pointer is NULL. > > who knows why? thx! It's possible that your standard C library implementation is not capable of handling large files. Read the documentation. You could be in a situation when you have to use OS-specific means for large file...

read from Text file using VB and Insert into SQL table until eof text file reach
Dear All, my text file like following, a1, b1, c1, d1, e1 a2, b2, c2, d2, e2 a3, b3, c3, d3, e3 a4, b4, c4, d4, e4 ................. and im going to insert this values into SQL Tables with columns, [a, b, c, d, e] how im going to do so? i use below src code to do but i can not get the next line and next line of text file until it finished insert Private Sub mot_Click() 'Shell ("C:\Program Files\Microsoft Office\Office11\MSACCESS.EXE c:\fisherdb. mdb /x ImportOpsNilai") 'frmPicCd.Show Dim db_connect ' Path = App.Path & "\ERP consumable code.xls" ...

Binary data stored in SQL Server: can't read from ASP.NET, *can* read from Access?
Working on converting an Access front-end/SQL Server back-end to a .NET front/SQL Server back-end. We are also redesiging the SQL Server database. We have many Word/Excel documents that have been stored as OLE Objects in the old SQL Server database. We can move them over to the new database okay. We can "read" the files okay when accessing them via Access (design mode/open table or runtime via double-clicking on an object frame directly bound to the appropriate column). However, we cannot read the files from ASP.NET. If we upload it from ASP.NET, we can read it fine. This lead...

How can I read X data from one file (.txt), Y data from another file (.txt) and plot a fit of the graph?
I have two files (.txt), one with the X values&nbsp;&nbsp;and the other (.txt)&nbsp;with the Y values. Both the files are like this: &nbsp; 34 54 76 89 ... &nbsp;and the number of rows is of course the same in both cases. &nbsp; I want to read them, find a fit for the curve and then have the values of the fit curve&nbsp;disposable for plotting&nbsp;them or for further elaboration. &nbsp; I have tried using "read from spreadsheet.vi" and the&nbsp;"curve fitting"&nbsp;block but I experienced several problems. &nbsp; What can I d...

Size of SAS file after reading from a raw text file
Hi, I have a text file of size close to 1.7 GB. it has five fields in it and 20373376 records. Im reading ALL of this data in to SAS using input statement and see that the final SAS file is just 473 MB. I have checked the number of observations in SAS and it is equal to the flat file and have also printed out some data and they seem to be fine. I expected SAS file to be bigger? (I have 3 other files having same kind of fields etc and for those the SAS data set is anywhere from 10% to 300% more than corresponding raw data) hp Depends on the data. By default, a numeric value takes eight...

Reading many fields/text/data from a single text file
I tried it with RTF, I am able to read the entire content. Same goes with the TXT file. But is VB able to allow you to read various data from a single text file? > I tried it with RTF, I am able to read the entire content. > Same goes with the TXT file. But is VB able to allow > you to read various data from a single text file? The only thing your post has told us is that you are having some kind of problem; but not the details of what that problem is. How big is the file you are trying to read? What is happening when you can't read the entire contents (for example, are you get...

Reading numerical data from a file which also contains a text data
Hello. I want to read numerical data from an output file, which also contains text data. The file test.t19 is as follows: id 1 =beg=50800 (Nodal Coordinates) 1 0.000000E+00 0.000000E+00 2 0.150000E+01 0.000000E+00 3 0.150000E+01 0.150000E+01 4 0.000000E+00 0.150000E+01 I want to read the coordinates of nodes 1~4. So what I did is as follows: snum = OpenRead["test.t19"] FindList[snum, "Coordinates"] ReadList[snum, Number, RecordLists -> True] The result was {{0}} ,though I expected {{1,0.00...

read mixed text, data file and find corresponding data values
Hello, could you help take a look my trouble for reading mixed text, data file. Thanks. I have many *.txt files (each file corresponding a condition) like: --------------------------start Text Line 1 --- Text line n Coordinates Point 1 X 9 Y 10 Z 8 Text Line 1 --- Text line n Coordinates Point 2 X 1 Y 10 Z 8 Force point 1 Direction X Y Z units Newton 10 20 30 ----------------------------end The number of text lines 'n' for different *.txt files is not the dame. However, the relative positions between the numeric value and its name flag, eg, 10 20 30 and Force, is fixed in al...

Question about reading a big binary file and write it into several text (ascii) files
Hi, I am learning and pretty new to Python and I hope your guys can give me a quick start. I have an about 1G-byte binary file from a flat panel x-ray detector; I know at the beggining there is a 128-byte header and the rest of the file is integers in 2-byte format. What I want to do is to save the binary data into several smaller files in integer format and each smaller file has the size of 2*1024*768 bytes. I know I can do something like >>>f=open("xray.seq", 'rb') >>>header=f.read(128) >>>file1=f.read(2*1024*768) >>>file2=f.read(2*10...

read data from a text file
Hello all, I have a text file and I want to read the data from it. I know I can do it by using getline or fread kind of functions. But the problem is data looks encoded in some unknown format and whenever I try to use getline or fread it doesn't show me anything. The data is in a text file called test.txt and it looks like following : A =C3=8E=154?=C3=8F=C3=91=03>=C2=A1=C2=90=07=C2=BF=C2=AD=C3=B4R>=C3= =A8=C2=A2=0B=C2=BE=C3=92=C2=AE=C3=BB=C2=BE=C3=B2=C3=B4=C2=81>~<=C3=A8=C2=BE= uE=C3=B0>=13u=C3=95=C2=BE=19=C3=87=C3=BB>=17=C2=9DA>=C2=8F=C3=B3=C2=B...

reading data from a text file
Hi: I have a problem to read input data in a text file. my input file looks like: % comments 11000 20 3 first line is a comment line having some characters, must be skipped second line contains three values that must be read individually, like variable a = 11000; b = 20; c = 3; please help me , how could I do this. Regards "Ali Haider" <alihaider82@gmail.com> wrote in message <hrlmhr$o8b$1@fred.mathworks.com>... > Hi: > > I have a problem to read input data in a text file. my input file looks like: > > % comments > 11000 20 3 ...

Reading data from a text file.
Hello, I am trying to read a 2d array from a text file. It is the lower half of a symmetric stiffness matrix. Below is a truncated matrix with 5 rows of the matrix. 4704236.44 , -3047252.386 , 4766438.599 799445.6067 , -1140470.3481 , 4450998.391 -579201.979 , 278200.066067 , 621983.5942 , 3727839.4485528 454987.0220 , -89773.15424 , -998275.316 , 383908.159 , 3839041.0909448 , The values are in the format [1,1] [2,1],[2,2 [3,1],[3,2],[3,3] [4,1],[4,2],[4,3],[4,4] [5,1],[5,2],[5,3],[5,4] [5,5] I am trying to use the "...

read data from text file
I am trying to read data from one file, get rid of the carriage return using the fgetl and print the new data into a new text file. My code is the following: % open text file fidData = fopen( 'xbee_hyp_term_data_6.txt' ); fidTXT = fopen( 'hyp_term_data_6.txt', 'wt' ); eofstat = feof(fidData); % x = 1; while ~feof(fidData) tline = fgetl(fidData); if ~ischar(tline), break, end fprintf(fidTXT, '%c', tline); % x = x+1; end A = fscanf(fidTXT,'%1s',inf) fclose(fidTXT) fclose(fidData) I am currently using the fscanf, but have tried...

How to read a Text File data
Hello, how can i read some text file data and using it in Matlab 7.0? Thank You, Miguel help save help lookfor "Miguel Dias Moura" <web001@27lamps.com> wrote in message news:eee638f.-1@webx.raydaftYaTP... > Hello, > > how can i read some text file data and using it in Matlab 7.0? > > Thank You, > Miguel help importdata help fopen help fread help textscan Michael "Miguel Dias Moura" <web001@27lamps.com> schrieb im Newsbeitrag news:eee638f.-1@webx.raydaftYaTP... > Hello, > > how can i read some text file data and using it in ...

reading data from text file
hi I'm new to C++ and I'm currently working on Visual C++ 6.0 I'm trying to open a text file, and read some data from it part of the text file looks like this: -------- Stoichiometry C3H7NO Framework group C1[X(C3H7NO)] Deg. of freedom 30 Full point group C1 NOp 1 Largest Abelian subgroup C1 NOp 1 Largest concise Abelian subgroup C1 NOp 1 Standard orientation: --------------------------------------------------------------------- Center Atomic Atomic Coordinates (Angstroms) N...

How can I read out an text file??
How can i read out a text file... For example &nbsp; text.txt.... &nbsp; I need to no what's the string in Line1, line2 etc &nbsp; Can I read out a text file line for line?? &nbsp; &nbsp; I've tried to do this but, &nbsp; I use te read out text file item..... Enable read lines.... Put on the count input 1 or 2.......&nbsp; and put that in an string indicator......... &nbsp; but it won't work......... i only see the first line of my text file in the string indicator........ &nbsp; It may be very easy but i can't solve this problem..... &n...

Reading data from a text file
I have a small text file which consist of the following data: [[[ -1 -1 -1 1 You are in a small square room. 0 -1 -1 -1 You are in a big round room. ]]] And the code I've written is as follows: [[[ Rooms getRoomDataRecords(string roomDataFile) { Rooms rooms; ifstream ifs(roomDataFile.c_str()); if (!ifs.is_open()) { cout << "Eh? Could not open " << roomDataFile << " for reading." << endl; return rooms; } while (!ifs.eof()) { int n, w, e, s; string line; ifs >> n >> w >> e >> s; cout << "...

Reading data from Text file
Hello, I have a problem on reading data from Text file(notepad) and save as a variable.Data that i want to get from text file are shown below ( sample1, 3, add).Below are java code just to get "sample1" put in classString. How can i modify the code to get "3" and "add". Can anyone help me and i also welcome with the java code. Thank you... ------------Text file (injector.txt)------------ classname = sample1 paramtypes = 3 methodname = add ------------------------------------------------- -------------Java source code--------------------- static String cl...

Read data in a text file
Hello, I need to extract data from a text file that I cannot modify. The text file is as following: Position across width (M1,S1);;-0.635;-0.623;-0.598; Exit Stress Deviation (MPa);MPa;140;-12.1;-12.1; I just want to extract the numerical data: -0.635 -0.623 -0.598 140 -12.1 -12.1 Does anybody know how to do it ? I know that there is a function called textread but I don't know how to set the options to keep the numerical data only. Thank you very much, Guillaume Hi, try this : [A,B,C]=textread('data.txt','%*s%*s%f%f%f','delimiter',';') J�r�me Hi...