I/O "mistery" or compiler bug?

  • Follow


Hi, all,

a coworker I am supervising has to work on legacy code and needs some
help. I post a short snippet :

parameter(iu = 12)
parameter(nval=1)
parameter(nmax=9)

integer ival, i
real  vector(nval,nmax)

open(iu,file='input.dat', status='old')
do i=1,nmax
         write(*,*) i
         read(iu,*,err=992) (vector(ival,i),ival=1,nval)
end do
stop

992 write(*,*) 'I/O error'

end

input.dat
0.546654681	! 1
0.340747334	! 2
0.690892203	! 3
0.463650087	! 4
0.718833868	! 5
0.878740016	! 6
0.317021327	! 7
0.677782867	! 8

0.716865424	! 9

The code looks innocuous: it compiles smoothly and does what's
expected under Windows with Intel(R) Fortran Compiler 10.1.025.
However, with xlf on a PowerPc, it arrives to line 8 and then writes
'I/O error', as if the empty line triggered the err=992 clause. How is
that possible?

Best Regards

Sergio Rossi
0
Reply deltaquattro (202) 7/17/2009 1:14:28 PM

deltaquattro wrote:
....[snip preliminaries for brevity]...

> open(iu,file='input.dat', status='old')
> do i=1,nmax
>          write(*,*) i
>          read(iu,*,err=992) (vector(ival,i),ival=1,nval)
> end do
> stop
> 
> 992 write(*,*) 'I/O error'
> 
> end
> 
> input.dat
> 0.546654681	! 1
....
> 0.677782867	! 8
> 
> 0.716865424	! 9
> 
> The code looks innocuous: it compiles smoothly and does what's
> expected under Windows with Intel(R) Fortran Compiler 10.1.025.
> However, with xlf on a PowerPc, it arrives to line 8 and then writes
> 'I/O error', as if the empty line triggered the err=992 clause. How is
> that possible?
....

Almost anything is _possible_... :)

Looks like a problem in xlf to me assuming there's nothing other than an 
empty record (that is, no invalid non-printing characters) in the input 
file.

A blank or premature end of record is, afaik, supposed to simply go on 
to the next record in list-directed i/o.  (That begs the question of why 
one would structure the input that way, but... :) )

--
0
Reply none1568 (6652) 7/17/2009 2:51:23 PM


dpb <none@non.net> wrote:

> deltaquattro wrote:
> ...[snip preliminaries for brevity]...
> 
> > open(iu,file='input.dat', status='old')
> > do i=1,nmax
> >          write(*,*) i
> >          read(iu,*,err=992) (vector(ival,i),ival=1,nval)
> > end do
> > stop
> > 
> > 992 write(*,*) 'I/O error'
> > 
> > end
> > 
> > input.dat
> > 0.546654681 ! 1
> ...
> > 0.677782867 ! 8
> > 
> > 0.716865424 ! 9
> > 
> > The code looks innocuous: it compiles smoothly and does what's
> > expected under Windows with Intel(R) Fortran Compiler 10.1.025.
> > However, with xlf on a PowerPc, it arrives to line 8 and then writes
> > 'I/O error', as if the empty line triggered the err=992 clause. How is
> > that possible?
> ...
> 
> Almost anything is _possible_... :)
> 
> Looks like a problem in xlf to me assuming there's nothing other than an
> empty record (that is, no invalid non-printing characters) in the input
> file.
> 
> A blank or premature end of record is, afaik, supposed to simply go on
> to the next record in list-directed i/o.  (That begs the question of why
> one would structure the input that way, but... :) )

I'd suggest removing the err= so as to see what xlf doesn't like. That
ought to give you an error message. As dpb says, there are lots of
possibilities. To throw out another, it is not 100% obvious that the
problem isn't on line 9 instead of 8; how about a missing end-of-record
mark for the last record, for example. I would not be willing to commit
to anything based on the data given.

-- 
Richard Maine
email: last name at domain . net
domain: summer-triangle
0
Reply nospam47 (9742) 7/17/2009 3:06:14 PM

deltaquattro wrote:
> 
> Hi, all,
> 
> a coworker I am supervising has to work on legacy code and needs some
> help. I post a short snippet :
> 
> parameter(iu = 12)
> parameter(nval=1)
> parameter(nmax=9)
> 
> integer ival, i
> real  vector(nval,nmax)
> 
> open(iu,file='input.dat', status='old')
> do i=1,nmax
>          write(*,*) i
>          read(iu,*,err=992) (vector(ival,i),ival=1,nval)
> end do
> stop
> 
> 992 write(*,*) 'I/O error'
> 
> end
> 
> input.dat
> 0.546654681     ! 1
> 0.340747334     ! 2
> 0.690892203     ! 3
> 0.463650087     ! 4
> 0.718833868     ! 5
> 0.878740016     ! 6
> 0.317021327     ! 7
> 0.677782867     ! 8
> 
> 0.716865424     ! 9
> 
> The code looks innocuous: it compiles smoothly and does what's
> expected under Windows with Intel(R) Fortran Compiler 10.1.025.
> However, with xlf on a PowerPc, it arrives to line 8 and then writes
> 'I/O error', as if the empty line triggered the err=992 clause. How is
> that possible?

Hmmm, mixed Windows/Mac environment.  Is there a carriage return (\r,
hex 0D) on line 8.5?  That would probably trigger this result with XLF
8.1 on Mac OS PPC.

If this is a one-shot application, just use a text editor with global
search/replace to remove all the returns.  Or else, on the command line:

  cat input.dat | tr -d '\r' > input2.dat

"There are more than two ways to do this."   ;-)

For a more routine and robust fix in Fortran, read each line first as a
wide character string, and manually remove the terminal return
character, if present.  Then do an internal read from the string to read
numbers.

This method works well for nice square matrices of numbers.  However, it
will need refinement if you expected single read statements to get
multiple items split across more than one line.

--Dave
0
Reply nospom (125) 7/17/2009 3:24:43 PM

On 17 Lug, 16:51, dpb <n...@non.net> wrote:
> deltaquattro wrote:
[..]
> Almost anything is _possible_... :)
>

Ah, ah, that's for sure, expecially in the magic world of
programming :)

> Looks like a problem in xlf to me assuming there's nothing other than an
> empty record (that is, no invalid non-printing characters) in the input
> file.
>

That's what I thought, too.

> A blank or premature end of record is, afaik, supposed to simply go on
> to the next record in list-directed i/o. =A0(That begs the question of wh=
y
> one would structure the input that way, but... :) )
>
> --

Here's the answer to your question: the original input looked more or
less like this,

0.546654681     ! 1 =3D=3D=3D=3DSTART OF SECTION 1=3D=3D=3D=3D=3D=3D
0.340747334     ! 2 ....
0.690892203     ! 3 ...
0.463650087     ! 4 ...
0.718833868     ! 5 ..
0.878740016     ! 6 ..
0.317021327     ! 7 ..
0.677782867     ! 8  =3D=3D=3D=3DEND OF SECTION 1=3D=3D=3D=3D=3D=3D=3D

0.716865424     ! 9  =3D=3D=3D=3DSTART OF SECTION 2=3D=3D=3D=3D=3D=3D
.....
.....
.....

Best,

Sergio


0
Reply deltaquattro (202) 7/17/2009 3:46:59 PM

On 17 Lug, 17:06, nos...@see.signature (Richard Maine) wrote:
> dpb <n...@non.net> wrote:
> > deltaquattro wrote:
> > ...[snip preliminaries for brevity]...
>
> > > open(iu,file=3D'input.dat', status=3D'old')
> > > do i=3D1,nmax
> > > =A0 =A0 =A0 =A0 =A0write(*,*) i
> > > =A0 =A0 =A0 =A0 =A0read(iu,*,err=3D992) (vector(ival,i),ival=3D1,nval=
)
> > > end do
> > > stop
>
> > > 992 write(*,*) 'I/O error'
>
> > > end
>
> > > input.dat
> > > 0.546654681 ! 1
> > ...
> > > 0.677782867 ! 8
>
> > > 0.716865424 ! 9
>
> > > The code looks innocuous: it compiles smoothly and does what's
> > > expected under Windows with Intel(R) Fortran Compiler 10.1.025.
> > > However, with xlf on a PowerPc, it arrives to line 8 and then writes
> > > 'I/O error', as if the empty line triggered the err=3D992 clause. How=
 is
> > > that possible?
> > ...
>
> > Almost anything is _possible_... :)
>
> > Looks like a problem in xlf to me assuming there's nothing other than a=
n
> > empty record (that is, no invalid non-printing characters) in the input
> > file.
>
> > A blank or premature end of record is, afaik, supposed to simply go on
> > to the next record in list-directed i/o. =A0(That begs the question of =
why
> > one would structure the input that way, but... :) )
>
> I'd suggest removing the err=3D so as to see what xlf doesn't like. That
> ought to give you an error message.

That's a good idea.

As dpb says, there are lots of
> possibilities. To throw out another, it is not 100% obvious that the
> problem isn't on line 9 instead of 8; how about a missing end-of-record
> mark for the last record, for example. I would not be willing to commit
> to anything based on the data given.

The original data was more or less like this:
input.dat
0.546654681     ! 1
....
.....
0.317021327     ! 7
0.677782867     ! 8

0.716865424     ! 9
....
....
0.83468342     ! 20

0.83468342     ! 21
.....
....

Every blank line triggered the err condition. I don't see why each
line before a blank line should  have a missing end-of.-record mark.
That's why I thought about the empty line, but surely that's no proof.
My coworker is out of office for a while, but as soon as he's back
I'll tell him to try your suggestion. Thank you very much,

Best Regards,

Sergio Rossi




>
> --
> Richard Maine
> email: last name at domain . net
> domain: summer-triangle

0
Reply deltaquattro (202) 7/17/2009 3:56:53 PM

Dave Allured wrote:
> 
> ...
> If this is a one-shot application, just use a text editor with global
> search/replace to remove all the returns.  Or else, on the command line:
> 
>   cat input.dat | tr -d '\r' > input2.dat
> 
> "There are more than two ways to do this."   ;-)

Sorry, I forgot:

  dos2unix input.dat input2.dat

Check syntax and availability.

--Dave
0
Reply nospom (125) 7/17/2009 4:05:17 PM

On 17 Lug, 17:24, Dave Allured <nos...@nospom.com> wrote:
[..]
>
> Hmmm, mixed Windows/Mac environment.

It's even worse: Linux was installed on the PowerPc, and the
installation is so "strange" that many apps such as GNOME do not work.

Is there a carriage return (\r,
> hex 0D) on line 8.5?  That would probably trigger this result with XLF
> 8.1 on Mac OS PPC.
>

I think so. Which (freeware) editor can I use to check that? vi on
Linux does not show any ^M character, which is to be expected since
the file was transferred by FTP in ASCII mode, so no ^M characters
should have been generated. On Windows my coworker uses PsPad, but I
don't know if that can show non-printing character.

> If this is a one-shot application, just use a text editor with global
> search/replace to remove all the returns.  Or else, on the command line:
>
>   cat input.dat | tr -d '\r' > input2.dat
>
> "There are more than two ways to do this."   ;-)

I know :-) that's another:

sed -e 's/\r$//'  input.dat > input.dat.noCtrlM

for the above reasons, I don't think that's the problem, but I'll be
sure to try your suggestion.

>
> For a more routine and robust fix in Fortran, read each line first as a
> wide character string, and manually remove the terminal return
> character, if present.  Then do an internal read from the string to read
> numbers.

Yes, I know: that's like I have done in a lot of my codes, and I
learned this technique thanks to you guys from comp.lang.fortran.
However, in my new job I don't code anymore directly, I just supervise
other colleagues. I suggested this coworker to modify the read this
way, but he is very reluctant to touch the code, since it's from an
external  contractor and he'd prefer to leave any modifications to the
contractor.

>
> This method works well for nice square matrices of numbers.  However, it
> will need refinement if you expected single read statements to get
> multiple items split across more than one line.
>
> --Dave

Sure, that would be quite trickier. Thanks to all of you guys,

Best Regards

Sergio


0
Reply deltaquattro (202) 7/17/2009 4:39:00 PM

On 17 Lug, 16:51, dpb <n...@non.net> wrote:
> deltaquattrowrote:
>
> ...[snip preliminaries for brevity]...
>
>
>
>
>
> > open(iu,file=3D'input.dat', status=3D'old')
> > do i=3D1,nmax
> > =A0 =A0 =A0 =A0 =A0write(*,*) i
> > =A0 =A0 =A0 =A0 =A0read(iu,*,err=3D992) (vector(ival,i),ival=3D1,nval)
> > end do
> > stop
>
> > 992 write(*,*) 'I/O error'
>
> > end
>
> > input.dat
> > 0.546654681 =A0 =A0 =A0 =A0! 1
> ...
> > 0.677782867 =A0 =A0 =A0 =A0! 8
>
> > 0.716865424 =A0 =A0 =A0 =A0! 9
>
> > The code looks innocuous: it compiles smoothly and does what's
> > expected under Windows with Intel(R) Fortran Compiler 10.1.025.
> > However, with xlf on a PowerPc, it arrives to line 8 and then writes
> > 'I/O error', as if the empty line triggered the err=3D992 clause. How i=
s
> > that possible?
>
> ...
>
> Almost anything is _possible_... :)
>
> Looks like a problem in xlf to me assuming there's nothing other than an
> empty record (that is, no invalid non-printing characters) in the input
> file.
>
> A blank or premature end of record is, afaik, supposed to simply go on
> to the next record in list-directed i/o. =A0(That begs the question of wh=
y
> one would structure the input that way, but... :) )
>
> --

Hi, dpb,

browsing through old post I found that once we discussed about a code
to read a file with an unknown amount of lines, and unknown amount of
"words" for line:

http://groups.google.it/group/comp.lang.fortran/msg/7f572b0a84d4aedc

may I ask to to post the code? I'd like very much to see it and learn
something more, even though probably I won't get to use it (see answer
to Dave Allured). Thanks,

Best Regards,

Sergio







0
Reply deltaquattro (202) 7/17/2009 4:46:25 PM

Dave Allured wrote:

> Dave Allured wrote:
> 
>>...
>>If this is a one-shot application, just use a text editor with global
>>search/replace to remove all the returns.  Or else, on the command line:
>>
>>  cat input.dat | tr -d '\r' > input2.dat
>>
>>"There are more than two ways to do this."   ;-)
> 
> 
> Sorry, I forgot:
> 
>   dos2unix input.dat input2.dat
> 
> Check syntax and availability.
> 
> --Dave

Heh.  I just got done recoding one of our legacy routines (reads 
formatted data/text files on Solaris boxes and processes the data into 
our database), so it could detect a Windoze-formatted file and 
automatically convert it to Unix formatting by stripping the CR 
characters off, instead of throwing an error message and telling the 
user to do the dos2unix as you show...  Getting to be more & more 
common, as our users nowadays often see the Unix box only as a mapped 
drive on their Windows PC and use any old editor to tweak these files.

Jim
0
Reply JCornwall (184) 7/17/2009 6:12:08 PM

deltaquattro wrote:

> On 17 Lug, 17:24, Dave Allured <nos...@nospom.com> wrote:
> [..]
> 
>>Hmmm, mixed Windows/Mac environment.
> 
> 
> It's even worse: Linux was installed on the PowerPc, and the
> installation is so "strange" that many apps such as GNOME do not work.
> 
> Is there a carriage return (\r,
> 
>>hex 0D) on line 8.5?  That would probably trigger this result with XLF
>>8.1 on Mac OS PPC.
>>
> 
> 
> I think so. Which (freeware) editor can I use to check that? vi on
> Linux does not show any ^M character, which is to be expected since
> the file was transferred by FTP in ASCII mode, so no ^M characters
> should have been generated. On Windows my coworker uses PsPad, but I
> don't know if that can show non-printing character.

Vim can show the ^M characters.  So can vi, at least the version we use. 
  I'm not certain about the command to show these, though.

Windows:  Textpad will show 'em, and gives you the "Save in Unix format" 
option.

Jim

> Best Regards
> 
> Sergio
> 
> 
0
Reply JCornwall (184) 7/17/2009 6:20:07 PM

On Jul 17, 6:14 am, deltaquattro <deltaquat...@gmail.com> wrote:
> Hi, all,
>
> a coworker I am supervising has to work on legacy code and needs some
> help. I post a short snippet :
>
> parameter(iu = 12)
> parameter(nval=1)
> parameter(nmax=9)
>
> integer ival, i
> real  vector(nval,nmax)
>
> open(iu,file='input.dat', status='old')
> do i=1,nmax
>          write(*,*) i
>          read(iu,*,err=992) (vector(ival,i),ival=1,nval)
> end do
> stop
>
> 992 write(*,*) 'I/O error'
>
> end
>
> input.dat
> 0.546654681     ! 1
> 0.340747334     ! 2
> 0.690892203     ! 3
> 0.463650087     ! 4
> 0.718833868     ! 5
> 0.878740016     ! 6
> 0.317021327     ! 7
> 0.677782867     ! 8
>
> 0.716865424     ! 9
>
> The code looks innocuous: it compiles smoothly and does what's
> expected under Windows with Intel(R) Fortran Compiler 10.1.025.
> However, with xlf on a PowerPc, it arrives to line 8 and then writes
> 'I/O error', as if the empty line triggered the err=992 clause. How is
> that possible?

Try removing the err= specifier from the READ statement.
If the program gives an error message at run time, please
let us know what it is.  Include the error message as
written by the program, not just a summary or paraphrase.

Bob Corbett
0
Reply robert.corbett2 (862) 7/18/2009 1:27:26 AM

deltaquattro wrote:
> 
> On 17 Lug, 17:24, Dave Allured <nos...@nospom.com> wrote:
> [..]
> >
> > Hmmm, mixed Windows/Mac environment.
> 
> It's even worse: Linux was installed on the PowerPc, and the
> installation is so "strange" that many apps such as GNOME do not work.

Why Linux?  My desktop at work is PPC with Mac OS, and I find it very
nice.  OTOH I am not a good judge because I have not spent any
significant time on Linux.

BTW, Linux and Mac OS both use line feed, i.e. newline, \n, hex 0A,
decimal 10 as the standard line terminator.  In other words they play
nicely together with text files.  On Windows the 2-char combination
CR/LF is traditional, which is why simply deleting all the returns works
so well for "importing" Windows text to Unix land.

> Is there a carriage return (\r,
> > hex 0D) on line 8.5?  That would probably trigger this result with XLF
> > 8.1 on Mac OS PPC.
> 
> I think so. Which (freeware) editor can I use to check that? vi on
> Linux does not show any ^M character, which is to be expected since
> the file was transferred by FTP in ASCII mode, so no ^M characters
> should have been generated. On Windows my coworker uses PsPad, but I
> don't know if that can show non-printing character.

Nedit, but that is suffering from lack of support lately.  Just use od
from the command line for a quick bytewise decode of the first few data
lines:

  od -Ad -c -N200 infile.dat      --Dumps characters as characters
  od -Ad -t x1 -N200 infile.dat   --Dumps characters as hex

Editors can try to outsmart you by hiding things.  od tells the truth
all the time.
 
> > If this is a one-shot application, just use a text editor with global
> > search/replace to remove all the returns.

I wish I had not said that, 'cuz now I remember that some text editors
try to hide the returns instead of letting you mess with them.  Even
nedit I think.  Go with tr (below) or dos2unix for easy, one-line return
stripping. 

Or else, on the command line:
> >
> >   cat input.dat | tr -d '\r' > input2.dat
> >
> > "There are more than two ways to do this."   ;-)
> 
> I know :-) that's another:
> 
> sed -e 's/\r$//'  input.dat > input.dat.noCtrlM
> 
> for the above reasons, I don't think that's the problem, but I'll be
> sure to try your suggestion.
> 
> >
> > For a more routine and robust fix in Fortran, read each line first as a
> > wide character string, and manually remove the terminal return
> > character, if present.  Then do an internal read from the string to read
> > numbers.
> 
> Yes, I know: that's like I have done in a lot of my codes, and I
> learned this technique thanks to you guys from comp.lang.fortran.
> However, in my new job I don't code anymore directly, I just supervise
> other colleagues. I suggested this coworker to modify the read this
> way, but he is very reluctant to touch the code, since it's from an
> external contractor and he'd prefer to leave any modifications to the
> contractor.

Well then, I think the politic thing would be to leave the code alone
for now, just return-strip a few input files for test cases to
demonstrate that you have identified the right problem.  Then ask the
contractor to add return stripping for you.  Best of luck there.

--Dave
0
Reply nospom (125) 7/18/2009 4:58:42 AM

On 18 Lug, 06:58, Dave Allured <nos...@nospom.com> wrote:
> deltaquattro wrote:
>
[..]
>
> Why Linux? =A0My desktop at work is PPC with Mac OS, and I find it very
> nice. =A0OTOH I am not a good judge because I have not spent any
> significant time on Linux.
>

Don't blame me, blame the SA guys ;-)

[..]
>
> I wish I had not said that, 'cuz now I remember that some text editors
> try to hide the returns instead of letting you mess with them. =A0Even
> nedit I think. =A0Go with tr (below) or dos2unix for easy, one-line retur=
n
> stripping.
>

I used dos2unix, and that went smooth! I was tricked by vi not showing
the ^M characters, probably due to some "strange" setting in .vimrc.

[..]
>
> Well then, I think the politic thing would be to leave the code alone
> for now, just return-strip a few input files for test cases to
> demonstrate that you have identified the right problem. =A0Then ask the
> contractor to add return stripping for you. =A0Best of luck there.

Right...I really don't like this scenario. I'd instruct my coworker to
do otherwise, but unfortunately the functional line manager agrees
with his view...Thank you guys for your help, as invaluable as always,

Best Regards

Sergio Rossi



>
> --Dave

0
Reply deltaquattro (202) 7/22/2009 7:31:35 AM

13 Replies
29 Views

(page loaded in 0.196 seconds)

Similiar Articles:


















7/28/2012 10:17:57 PM


Reply: