textscan - how to read empty rows?

  • Permalink
  • submit to reddit
  • Email
  • Follow


Dear Community,
I want to read text files that are all of the same structure (300 rows, text and numbers). Sometimes rows are empty but still they are there to make up the 300 in the original file. Using textscan and %s doesn't read the empty rows and thus reduces the total number of rows by the number of empty rows. Since I need to read specific data from specific row numbers I end up with data from the wrong rows when there are empty rows. I realize I should include something to read the delimiter to let textscan read the empty row (there must be a carriage return or something but I can't see it). How can I read the file to always get 300 lines even when there are empty lines.

Here are a couple of lines from the data file:

1.00
xtd_//1.000
0.00

10
20
5
042/1*371*/#13905/

this is my code: 

for k = 1:numberoffiles
    actualname=listoffiles(k).name;
    fopen (actualname);
    file_id = fopen(actualname);
    ProcessFile(k)=textscan(file_id, '%s');
    fclose (file_id);
end

and this is the result:

1.00
xtd_//1.000
0.00
10
20
5
042/1*371*/#13905/

If I want to automatically process the ProcessFile, things don't work because the number of rows differs from one column to the next because of the empty rows in the original files. Any helping comment welcome! 
Chris
0
Reply no4413 (1) 3/13/2010 4:55:08 PM

See related articles to this posting


"Christian " <no@newsgroup.com> wrote in message <hngg1c$ohq$1@fred.mathworks.com>...
> Dear Community,
> I want to read text files that are all of the same structure (300 rows, text and numbers). Sometimes rows are empty but still they are there to make up the 300 in the original file. Using textscan and %s doesn't read the empty rows and thus reduces the total number of rows by the number of empty rows. Since I need to read specific data from specific row numbers I end up with data from the wrong rows when there are empty rows. I realize I should include something to read the delimiter to let textscan read the empty row (there must be a carriage return or something but I can't see it). How can I read the file to always get 300 lines even when there are empty lines.
> 
> Here are a couple of lines from the data file:
> 
> 1.00
> xtd_//1.000
> 0.00
> 
> 10
> 20
> 5
> 042/1*371*/#13905/
> 
> this is my code: 
> 
> for k = 1:numberoffiles
>     actualname=listoffiles(k).name;
>     fopen (actualname);
>     file_id = fopen(actualname);
>     ProcessFile(k)=textscan(file_id, '%s');
>     fclose (file_id);
> end
> 
> and this is the result:
> 
> 1.00
> xtd_//1.000
> 0.00
> 10
> 20
> 5
> 042/1*371*/#13905/
> 
> If I want to automatically process the ProcessFile, things don't work because the number of rows differs from one column to the next because of the empty rows in the original files. Any helping comment welcome! 
> Chris

First read the lines as strings, using '\n' as a delimiter, thereby including the empty lines. Then process these strings individually, using a loop or CELLFUN:

S = textread('DATA.TXT','%s','delimiter','\n') ;
C = cell(numel(S),1)
for k=1:numel(S), 
  C{k} = sscanf(S{k},'%f') ; 
end

hth
Jos
0
Reply 10584 (932) 3/14/2010 6:05:08 PM

"Jos (10584) " <#10584@fileexchange.com> wrote in message <hnj8gj$bfj$1@fred.mathworks.com>...
> "Christian " <no@newsgroup.com> wrote in message <hngg1c$ohq$1@fred.mathworks.com>...
> > Dear Community,
> > I want to read text files that are all of the same structure (300 rows, text and numbers). Sometimes rows are empty but still they are there to make up the 300 in the original file. Using textscan and %s doesn't read the empty rows and thus reduces the total number of rows by the number of empty rows. Since I need to read specific data from specific row numbers I end up with data from the wrong rows when there are empty rows. I realize I should include something to read the delimiter to let textscan read the empty row (there must be a carriage return or something but I can't see it). How can I read the file to always get 300 lines even when there are empty lines.
> > 
> > Here are a couple of lines from the data file:
> > 
> > 1.00
> > xtd_//1.000
> > 0.00
> > 
> > 10
> > 20
> > 5
> > 042/1*371*/#13905/
> > 
> > this is my code: 
> > 
> > for k = 1:numberoffiles
> >     actualname=listoffiles(k).name;
> >     fopen (actualname);
> >     file_id = fopen(actualname);
> >     ProcessFile(k)=textscan(file_id, '%s');
> >     fclose (file_id);
> > end
> > 
> > and this is the result:
> > 
> > 1.00
> > xtd_//1.000
> > 0.00
> > 10
> > 20
> > 5
> > 042/1*371*/#13905/
> > 
> > If I want to automatically process the ProcessFile, things don't work because the number of rows differs from one column to the next because of the empty rows in the original files. Any helping comment welcome! 
> > Chris
> 
> First read the lines as strings, using '\n' as a delimiter, thereby including the empty lines. Then process these strings individually, using a loop or CELLFUN:
> 
> S = textread('DATA.TXT','%s','delimiter','\n') ;
> C = cell(numel(S),1)
> for k=1:numel(S), 
>   C{k} = sscanf(S{k},'%f') ; 
> end
> 
> hth
> Jos


This works fine, thanks Jos
0
Reply Christian 3/18/2010 9:23:04 PM
comp.soft-sys.matlab 203682 articles. 536 followers. Post

2 Replies
328 Views

Similar Articles

[PageSpeed] 57


  • Permalink
  • submit to reddit
  • Email
  • Follow


Reply:

Similar Artilces:

Having issues reading excle with some empty rows
I am trying to read excel file using POI with code which looks as follows HSSFWorkbook wb = new HSSFWorkbook(stream, false); HSSFSheet sheet = wb.getSheetAt(sheetNo); int noOfRows = sheet.getPhysicalNumberOfRows(); for (int r = 0; r < noOfRows; r++) { HSSFRow row = sheet.getRow(r); if(row == null) break; //read data } What happens is if say I have an excel with 10 rows then i select the data in last 5 row and delete it and save the excel. Now if I read th...

textscan a textfile and read the row that starts with a certain string
hello I need to read from a textfile with 400 rows code name regime start end surface altitude region regname A0212 Avelo PLUVI 01FEB 31JAN 23 344 -4 Tredor B0156 Grets NIVAL 01MAY 30AVR 51 1802 5 Fazal .... .... I just need to read one row which starts with a specific string. Say I want to read the line that starts with B0156 and that's it. i've tried: fid = fopen('info.txt'); C = textscan (fid, '%s %s %s %s %s %f %f %f %s', 'delimiter', '\t', 'headerlines', 1) fclose(fid) but I don't...

rows read vs. rows selected.
Hi All: I have a query which is running against large table. The query: SELECT DSRC_ACCT_ID, ADDR1, ADDR2, ADDR3, CITY, STATE, POSTAL_CODE, COUNT(*) FROM ERD.ADDRESSA GROUP BY DSRC_ACCT_ID, ADDR1, ADDR2, ADDR3, CITY, STATE, POSTAL_CODE HAVING COUNT(*) > 2 The applcation snapshot monitor shows: Rows selected = 181 Rows read = 646485498 Rows written = 933743357 The rows read is too high compared to rows selected. (The table has 933743360 rows) My questions is: What's the best...

Create empty dataset and populate it row by row
Hi, I'm trying to create an object of the class dataset (Stats toolbox) with a few named "columns". I want such object to initially have no rows. Then, I would like to add new rows one by one. I've tried the code below to create the dataset but I get an error. Can anyone help me with this? Thanks in advance P.S. I could not submit this question to the "answers" section of the Mathworks website % Column names are 'F1' and 'F2' myDS = dataset([],'VarNames',{'F1','F2'}) ??? Error using ==> setvarnames a...

Newbie: fastcsv: Read rows, print rows
New to Ruby. It probably shows. Trying to accomplish some simple initial tasks, and study. Objectives 1. Read rows from a CSV file 2. Write rows to STDOUT A cat() equivalent. I've been looking but don't find it. Are there any good tutorials on using fastcsv? This might be a good candidate for such. Many thanks! require 'fastercsv' fastercsv.open("rfile2", "r") do |csv| csvData.each{|row| puts "row: ${row}"} end On Jun 29, 2007, at 1:50 PM, drubdrub@gmail.com wrote: > New to Ruby. It probably shows. Welcome to Ruby! > ...

Read from Spreadsheet File.vi reading from a specific row
hi everyone, &nbsp; I am using Read from Spreasheet File.vi to read csv files. I want to start reading the data from the 4th row everytime (there are useless headers in the first 3 rows). But the "start of read offset" lets me count characters instead of number of rows. My files have random number of characters before the 4th row so there is no way i can use this function. Is there any other way to read everything starting from the 4th row? &nbsp; &nbsp; Thanks all, &nbsp; Tai Hi BestPker, I would suggest you read all the data first and then modify the "raw"...

DataGridView leaves an empty row after escape pressed on new row
Hello, I have a datagridview bound to a database table with 2 columns. One is an ID column "NameID" which is hidden, the other is called "Name". The schema picture is here : http://img89.imageshack.us/img89/1251/dbschemaep9.jpg What happens is : 1. Form loads with values from database inside the DGV 2. I press the checkbox to enable editing/adding etc in the grid (see code event) Picture : http://img145.imageshack.us/img145/8819/step1yy3.jpg 3good. I click inside the NewRow and type "a" or whatever (something invalid) ... Picture : http://img80.imageshack.u...

Re: Reading the data from the excel file into SAS by row by row
syam1919@GMAIL.COM wrote: >I am new to SAS Macro's world and currently working on SAS Macros. > >I want to read name of the file from the excel file into SAS which is >in the following format: > >Campaign_No Mail Data Socred File Quotes Files >1 Maildata_1 scored_1 Quotes_1 >2 Maildata_2 scored_2 Quotes_2 >3 Maildata_3 scored_3 Quotes_3 > >First I have to read file names contained in the first row, then second >row, then thi...

Delete row of cell array where any one cell in the row is empty or NaN
Hi I know I should know this, but I just can't get the right syntax :-( Given a={'one', [],'three';[],'five','six';'seven','eight',[];'ten','eleven','twelve';[],[],'fifteen'} a = 'one' [] 'three' [] 'five' 'six' 'seven' 'eight' [] 'ten' 'eleven' 'twelve' [] [] 'fifteen' What is the syntax to delete any row that contains at least one em...

Reading the data from the excel file into SAS by row by row and cell by cell
I am new to SAS Macro's world and currently working on SAS Macros. I want to read name of the file from the excel file into SAS which is in the following format: Campaign_No Mail Data Socred File Quotes Files 1 Maildata_1 scored_1 Quotes_1 2 Maildata_2 scored_2 Quotes_2 3 Maildata_3 scored_3 Quotes_3 First I have to read file names contained in the first row, then second row, then third row......... For example, in campaign_no:1, I have to get the file name Maildata_1, then Scored_1, Quotes_1 one by one.......... A...

Read Text row by row, plot them and find the max of its Power spectrum
Hi, everyone, &nbsp; I have to read data from a text file,&nbsp;plot as a&nbsp;time serie and find the frequency to its max power spectrum. I found a previous post related to my issue so I revised his example a little bit. But once I decide to store the data into an array, the Labview&nbsp;reported:' Error 1 occurred at Scan From String (arg 1)'. Can anyone help me out? &nbsp; Thanks in Advance! &nbsp; Gab ReadXYZmine.vi: http://forums.ni.com/attachments/ni/170/312601/1/ReadXYZmine.vi Hi Gab, the error pops up because the last two lines are empty. I have modi...

Re: Reading the data from the excel file into SAS by row by row #2
syam, You never create a macro var IND hence sas can't find one, hence the message you are recieving. You do however create a datastep variable Ind (via the do loop index var). If that is what you are wanting to assign to the variable INDI then best fix I think would be to change your do loop index var to INDI. Toby Dunn From: syam <syam1919@GMAIL.COM> Reply-To: syam <syam1919@GMAIL.COM> To: SAS-L@LISTSERV.UGA.EDU Subject: Re: Reading the data from the excel file into SAS by row by row and cell by cell Date: Tue, 6 Dec 2005 08:28:41 -0800 I have written thi...

Re: Reading the data from the excel file into SAS by row by row #3
In addition to the good points raised by Toby and David ... I see no connection between the code shown and the stated subject (Excel, rows, columns). On Tue, 6 Dec 2005 08:28:41 -0800, syam <syam1919@GMAIL.COM> wrote: >Hi All, > >I have written this code but I am facing little problems with this can >anybody suggest upon this; > > >%LET CSYAM_LIBLOC = 'C:\data\syam\'; >%LET CDRIVE_LIBLOC = 'C:\data\mail id data\'; > >LIBNAME CSYAM &CSYAM_LIBLOC; >LIBNAME CDRIVE &CDRIVE_LIBLOC; > >%MACRO NUM_OBSER(START= ,END= ); > > ...

table appears with empty rows, and row data appears before table. Code seems ok. What is going on?
I have code which displays a HTML <table>. The data in the rows comes is re= ad from a file. But the row data is appearing (minus the <tr></tr> tags) on= the web page before the table. The table displays with empty rows below wh= at was supposed to be the row data. In other words this php... echo "<table>\n<thead>\n<tr><th>number</th><th>date</th><th>First Name</th>= <th>Last Name</th><th>state</th><th>country</th><th>remarks</th></tr>\n</th= ead>\n<tbo...

creating empty rows
hi, i have a data set ,as per the sap i need to insert some empty rows to get the final report format. final report s.no name age s.no name age 1 ssn 25 1 ssn 25 2 robin 45 empty row 3 raju 56 2 robin 45 4 mike 65 3 raju 56 5 dell 76 empty row 4 mike 65 5 dell 76 __________________________________________________________ Yahoo! India Matrimony: Find your partner now. Go to http://yahoo.shaadi.com Here is one approach: /* oh how i wish folks would submit a work...

Reading empty File
Hi All I have an empty file : /mnt/ramdisk0/js01 I open it with --> f_send01 = File.open("/mnt/ramdisk0/js01","w+") and next line is --> l_send01 = f_send01.readline and rubby give error msg EOFError. Question : How to tell ruby to go to next command if only the file is not empty. well basically what i want to do is : -- open a file -- if not empty , output the content , and make the file empty regards -bino- -- Posted via http://www.ruby-forum.com/. > I open it with --> f_send01 = File.open("/mnt/ramdisk0/js01","w+") I could be ...

Read a textfile with textscan
Hi. I need to read a text file by lines, storing the result in a cell array of string like the textread function does. The problem is each line is compound of various strings separated by whitespaces, and I want to retrieve the whole content of each line, not as separated strings. I think it can't be done with textread so I've tried with textscan. But now the problem is that texscan returns a cel array and I need a cell array of strings. How could I convert it to an array of strings? Thanks a lot. "Dani " <asdf@asdf.com> wrote in message <g1dnd2$t67$1@fr...

textscan not reading all the values
Hello, I am using textscan to read a text file that has hundreds of lines and each value in a line delimited by a space. To be precise, these are the commands i'm using: >> fid=fopen('m5nonreas_m5nonreas3gm.txt'); >> c=textscan(fid,'%s %s %f %s %f %s %f %s %s %s %s %f %s %f %s %f','delimiter',' '); but its only reading the 25 lines. >> size(c{1}) ans = 25 1 Can anybody point out whats wrong? Thanks a lot Mikesh Mikesh Udani wrote: > Hello, I am using textscan to read a text file that has hundreds of ...

how to read data by textscan
the data.text is like the following: 11/12 Cas,29.1,30.52 Nas, 20,44 ALA,11.303,33.2 33/24 NuMA, 22,33.2 MDS, 222,200 ...... The number of column is different for each row. So how to read the text by textscan? I have did it like: n=1; while(~feof(fid)) fid=fopen(data.text); temp=textscan(fid,'%s',1); ID(n,1)=temp{1}; temp=textscan(fid,'%s %f %f ', 'delimiter',','); Name(n,1)=temp{1,1}; x(n,1)=temp{1,2}; y(n,1)=temp{1,3}; n=n+1; end But the result is wrong. I was comfused with how to use textscan. Any help is ...

read in words as rows
I have a big text file with long rows and I want to read in each word on a separate line into 1 single variable. file: I have a big text file with long rows and I want to read in each word on a separate line data set: I have a big text file with .... you get the idea, what option do i use for this? Dirk ...

Reading datafile with textscan
I am doing finite element postprocessing with ML and have a series of datafiles--one for each time step in the solution. My code reads the data from these .txt files using the textscan: [nodes]=textscan(fid,'%f %f %f %f',numnode,'headerlines',headers); I'm checking runtimes now and loading the data definitely dominates the code. Any suggestions for speeding this up? Do the files always have the same number of header lines? Are these necessary, or can you create the text files with just the data? In general, the more sophisticated the function, the longer it takes to ru...

textscan
Dear all, I am trying to read a file with the following format: PG01 -14863.279160 -507.286354 -21817.822342 67.103466 10 9 6 137 for this format i simply used (accoring to the fields which intresting me): textscan( FID , 'PG%2f %*14.6f%*14.6f%*14.6f%14.6f %*2f %*2f %*2f %3f %6' ) my problems started when i was facing the following line: PG27 5495.515924 15607.701735 21322.221168 59.491399 8 4 139 which mean, the first value that %*2f was suppose to read is missing and it reads the "second" value. In other words %3f returns me NaN (because there is noth...

Empty Rows on Report
I found some code that works great to creat empty rows of boxes on a report from the end of the data to the bottom of the report. Now, I can't seem to find the original message to post a question on that thread. The code is as follows. In the Report OnFormat section the [Event Procedure] I use the following. Private Sub Report_Page() Dim intBox As Integer Dim intRow As Integer Dim ctrl As Rectangle Dim lngDetTop As Long Dim lngDetHeight As Long lngDetTop = Me.Section(5).Height lngDetHeight = Me.Section(0).Height For intBox = 0 to 11 For intRow = 0 to 31 Set ct...

Read with empty lines
Hi there, I have a three-column data file containing coordinates to draw the entire Dutch coastline. The coastline is drawn in seperate lines for e.g. small islands etc. The seperate lines forming the coastline are seperated by empty lines in the data file. If I use a textread command to read the data the empty lines are ignored and this leads to one line where all islands are connected if I plot it. What I would like is to read e.g. the x-coordinates into a matrix X and start a new column in this matrix each time an empty line is encountered in the input file. So my question is if anyone can...