I'm trying to read in data in the form of a BIG cell array of strings (2million lines).
A few lines for the example:
'"7";"1";"01-01-2008 07:00:00";"0,0000"'
'"8";"1";"01-01-2008 08:00:00";"0,0000"'
'"9";"1";"01-01-2008 09:00:00";"8639898,0000"'
'"10";"1";"01-01-2008 10:00:00";"10113328,6400"'
If I use strtok, it takes about 10 minutes. Is there a faster way to do this? I read here that strread might be the solution, but I cannot get it to work properly.
My best shot is something like this:
[nIndex,nStreet,sDate{:},nEnergy1,nEnergy2] = strread(a{:},'"%u" "%u" %q "%d,%u"',10,'delimiter',';');
What I would like to have:
nIndex : integer array of index values
nStreet: integer array of street values
sDate: datenumbers, but cell array of date-strings is also OK
nEnergy: energy value. But in Dutch we use the comma as the decimal separator..
As for the dates, 23 hour values are always like this:
datenum(sDate{i}, 'dd-mm-yyyy HH:MM:SS')
and the 24th one is alway like this:
datenum(sDate{i}, 'dd-mm-yyyy')
Most importantly, i'd like strread to work.
|
|
0
|
|
|
|
Reply
|
Lennart
|
12/3/2009 12:15:24 PM |
|
"Lennart van Luijk" <MyFirstNameHere.van.luijk@gmail.com> wrote in message <hf8a4s$t$1@fred.mathworks.com>...
> I'm trying to read in data in the form of a BIG cell array of strings (2million lines).
> A few lines for the example:
>
> '"7";"1";"01-01-2008 07:00:00";"0,0000"'
> '"8";"1";"01-01-2008 08:00:00";"0,0000"'
> '"9";"1";"01-01-2008 09:00:00";"8639898,0000"'
> '"10";"1";"01-01-2008 10:00:00";"10113328,6400"'
>
> If I use strtok, it takes about 10 minutes. Is there a faster way to do this? I read here that strread might be the solution, but I cannot get it to work properly.
> My best shot is something like this:
>
> [nIndex,nStreet,sDate{:},nEnergy1,nEnergy2] = strread(a{:},'"%u" "%u" %q "%d,%u"',10,'delimiter',';');
>
> What I would like to have:
> nIndex : integer array of index values
> nStreet: integer array of street values
> sDate: datenumbers, but cell array of date-strings is also OK
> nEnergy: energy value. But in Dutch we use the comma as the decimal separator..
>
> As for the dates, 23 hour values are always like this:
> datenum(sDate{i}, 'dd-mm-yyyy HH:MM:SS')
> and the 24th one is alway like this:
> datenum(sDate{i}, 'dd-mm-yyyy')
>
> Most importantly, i'd like strread to work.
Here is suggestion:
% your cell array of strings
C= {'"7";"1";"01-01-2008 07:00:00";"0,0000"' ;
'"8";"1";"01-01-2008 08:00:00";"0,0000"' ;
'"9";"1";"01-01-2008 09:00:00";"8639898,0000"' ;
'"10";"1";"01-01-2008 10:00:00";"10113328,6400"'}
% make into a single long string
C2 = sprintf('%s;',C{:}) ;
% replace commas by dots
C2 = strrep(C2,',','.') ;
% read, using the appropriate symbols as delimiter and whitespace
[nIndex, nStreet,sDate,nEnergy] = strread(C2,'%d%d%s%f','delimiter',';','whitespace',' "')
hth
Jos
|
|
0
|
|
|
|
Reply
|
Jos
|
12/3/2009 12:45:08 PM
|
|
Thanks for the tip,
but this gives me an Out Of Memory error on a laptop with windows XP and 4GB of RAM. Also after i use the 'pack' command.
The procedure seems to work for smaller sets of data.
Any other ideas?
|
|
0
|
|
|
|
Reply
|
Lennart
|
12/3/2009 2:18:20 PM
|
|
"Lennart van Luijk" <MyFirstNameHere.van.luijk@gmail.com> wrote in message <hf8hbc$sus$1@fred.mathworks.com>...
> Thanks for the tip,
>
> but this gives me an Out Of Memory error on a laptop with windows XP and 4GB of RAM. Also after i use the 'pack' command.
>
> The procedure seems to work for smaller sets of data.
> Any other ideas?
If you read help in strread you will notice info that the textscan function is intended replace strread (textread). One of the issuse is memory problem and other is that strread is time costing.
Try to use textscan function instead.
Branko
|
|
0
|
|
|
|
Reply
|
Branko
|
12/3/2009 3:13:04 PM
|
|
|
3 Replies
283 Views
(page loaded in 0.059 seconds)
Similiar Articles: Strread String and floating point number problem! - comp.soft-sys ...... alpha(i)] = strread( line, '%f %f' ); and it doesn't like it either. Any suggestions on how to get this working ... Hi John, I can seem to only get this to partially work ... handling multiple lines of input from inputdlg - comp.soft-sys ...... answer{1,1} gives me: ans = test second I can't figure out how to handle this output. Attempting to view it as an array doesn't work, neither does strread. Adding an extension to arcmap trouble - comp.soft-sys.gis.esri ...I'm working on a class project. The instructor gave us the hydrologic extension from ESRI. I can't get it to work Here is the instructions I got:... How do I copy recordsets from one database table to another ...I would like to do it directly with the data in variant form, but I can't get it to work. I can get it to work(on a small test table) if I first convert my source ... ASA Hairpin routing w/NAT - comp.dcom.sys.ciscoHas anyone gotten the hairpin routing to work w/NAT? I currently have an old vpn concentrator and a pix515(v6 code) that we are hoping to replace wit... Custom Properties linking to filename. - comp.cad.solidworks ...Hi: There was a thread regarding this subject back in May. I tried to reply to it but couldn't get it to work. Here is a link to the original thre... for implicit heat equation - comp.soft-sys.matlabHi everyone, I am trying to create the codes for Implicit Heat Equation. However, I can't seem to get it work (I can't get the same dimension for th... Write to stderr? - comp.lang.rexxI'm trying to write to stderr from rexx, but can't get it to work - I just end up with a bunch of files called "stderr" or however I try to reference ... self intersecting loft? - comp.cad.solidworksIt does touch at a single edge but I still can't get it to work. >Intuitively, the surface that's created can't create an invalid > solid, if you're making a solid. NASM: section start address - comp.lang.asm.x86In fact, I can't get it to NOT work. So I don't know what I was doing wrong. It was a while back, so I can't remember exactly what I was trying to do (and I obviously ... Strread String and floating point number problem! - comp.soft-sys ...... alpha(i)] = strread( line, '%f %f' ); and it doesn't like it either. Any suggestions on how to get this working ... Hi John, I can seem to only get this to partially work ... "STRREAD will be removed in a future relea... - Newsreader ...can't see how simply supporting strread() going forward could possibly be in that category as nice as ... "MATLAB is a language with good syntax for working with matrices, a ... 7/28/2012 4:22:38 PM
|