IOSTAT for input from sequential formatted files with "incomplete" final record

  • Follow


Consider a file, that is not of fortran I/O origin, that does not have 
an end of record maker prior to the end of file:

Line 1<eor>
Line 2<eof>

where <eor> is the end of record marker, that also happens to be the 
line terminator for your platform of choice, and <eof> just means end of 
file (which may or may not involve some sort of explicit marker).

(Basically - I've got a text file that doesn't end in a newline on a 
unix-like or windows platform).

Then consider the following program:

PROGRAM eieio_stat
   IMPLICIT NONE
   INTEGER, PARAMETER :: unit = 10
   INTEGER :: ios1, ios2, ios3
   CHARACTER(100) :: buffer
   !****
   OPEN(UNIT=unit, FILE='file3.txt')  ! file 'xxx' has contents as above
   READ (unit,"(A)",ADVANCE="NO",iostat=ios1) buffer
   READ (unit,"(A)",ADVANCE="NO",iostat=ios2) buffer
   READ (unit,"(A)",ADVANCE="NO",iostat=ios3) buffer
   CLOSE(unit)
   WRITE (*, "(3(A,'(',I0,')',:,', '))")  &
       ioch(ios1), ios1, ioch(ios2), ios2, ioch(ios3), ios3
CONTAINS
   FUNCTION ioch(iostat)
     USE, INTRINSIC :: ISO_FORTRAN_ENV, ONLY: IOSTAT_END, IOSTAT_EOR
     INTEGER, INTENT(IN) :: iostat
     CHARACTER(4) :: ioch
     !****
     IF (iostat == IOSTAT_END) THEN
       ioch = 'IOSTAT_END'
     ELSE IF (iostat == IOSTAT_EOR) THEN
       ioch = 'IOSTAT_EOR'
     ELSE IF (iostat < 0) THEN
       ioch = 'Other -ve '
     ELSE IF (iostat > 0) THEN
       ioch = 'Positive  '
     ELSE
       ioch = 'Zero      '
     END IF
   END FUNCTION ioch
END PROGRAM eieio_stat

On Intel Fortran I get:
IOSTAT_EOR(-2), IOSTAT_EOR(-2), IOSTAT_END(-1).

On very recent gfortran 4.5.0 I get:
IOSTAT_EOR(-2), Zero      (0), IOSTAT_END(-1).

On g95 I get:
IOSTAT_EOR(-2), Positive  (207), Positive  (207).

These differences cause issues with the get procedure from 
iso_varying_string to read such "missing final newline" files and I've 
been trying to understand what the best/most robust way forward is 
(putting aside STREAM access for now).

When attempting to rationalise the results, some of my conflicting 
thoughts have been:

- Because the input file was not written by Fortran as a sequential 
formatted file (the way I am trying to read it) and misses the formally 
required end of record marker, then "anything goes" so I'm sunk and this 
whole post is irrelevant.

- Intel's treatment is probably the most useful, the second EOR can be 
justified because the record did end (though it may have been missing a 
formal marker it's up to the processor to decide what the end of a 
record is?) and some useful stuff can be retrieved into buffer.

- G95's treatment is not terribly useful, but still formally correct as 
the lack of an end of record marker is an error condition.  From that 
point on the unit is hosed.

- gfortran's treatment is also useful, but a little inconsistent - 
something terminated the input into buffer, if it wasn't the end of 
record, end of file or some other error condition (ie. record 
incomplete), then what was it?

By useful, I mean that I can get access to the text that is on the 
troublesome last line.

Thoughts?

Thanks,

IanH
0
Reply ian_harvey (217) 2/3/2010 8:14:24 AM

Ian Harvey <ian_harvey@bigpond.com> wrote:

> (Basically - I've got a text file that doesn't end in a newline on a 
> unix-like or windows platform).
....
> These differences cause issues with the get procedure from 
> iso_varying_string to read such "missing final newline" files and I've
> been trying to understand what the best/most robust way forward is 
> (putting aside STREAM access for now).

I wonder why you are putting aside stream? That is the only way where
the standard gives any help in the matter. As long as you open the file
as a sequential formatted file, then I think you are stuck with...

> - Because the input file was not written by Fortran as a sequential 
> formatted file (the way I am trying to read it) and misses the formally
> required end of record marker, then "anything goes" so I'm sunk and this
> whole post is irrelevant.

The bit about not being written by Fortran perhaps overstates the
requirements, but the file does have to be one that is valid by whatever
rules the Fortran compiler (ok, "processor" in standard-speak) defines.
The standard is silent on what those rules are. In practice, you are
essentially guaranteed that all will be ok if the files end in the
platform's newline sequence (for the mentioned platforms, which have
one). The standard doesn't guarantee that, but practice does.

Contributing to the "anything goes" is "the standard doesn't specify
what counts as an error condition."

> By useful, I mean that I can get access to the text that is on the 
> troublesome last line.

For "improperly structured" files like the ones in question, you can
debate what you feel would be useful, and perhaps try to talk vendors
into doing it, but the standard doesn't offer any support.

I would think it more constructive to pursue using what is already in
the standard (stream) instead of trying to push vendors into defacto
standardization of something that isn't formally standard and, by your
own evidence, isn't handled consistently by different existing
compilers. You are, of course, free to try, but my guess is that such a
request wouldn't be considered very high prority when a standard
conforming approach already exists. It also has the problem that you
might have to go through it again with every new vendor that becomes
relevant to you.

-- 
Richard Maine                    | Good judgment comes from experience;
email: last name at domain . net | experience comes from bad judgment.
domain: summertriangle           |  -- Mark Twain
0
Reply nospam 2/3/2010 8:50:50 AM


In article <1jdbapv.1ad8d151o18ny2N%nospam@see.signature>,
 nospam@see.signature (Richard Maine) wrote:

> For "improperly structured" files like the ones in question, you can
> debate what you feel would be useful, and perhaps try to talk vendors
> into doing it, but the standard doesn't offer any support.

There are many unix utilities that will have problems with that last 
unterminated line, not just fortran compilers and fortran i/o 
libraries.  I would suggest shifting the emphasis back one step and 
try to fix the offending program that creates such files.

$.02 -Ron Shepard
0
Reply Ron 2/3/2010 4:58:05 PM

"Ian Harvey" <ian_harvey@bigpond.com> wrote in message news:Afaan.5370$pv.135@news-server.bigpond.net.au...
| Consider a file, that is not of fortran I/O origin, that does not have
| an end of record maker prior to the end of file:
|
| Line 1<eor>
| Line 2<eof>
|
| where <eor> is the end of record marker, that also happens to be the
| line terminator for your platform of choice, and <eof> just means end of
| file (which may or may not involve some sort of explicit marker).
|
| (Basically - I've got a text file that doesn't end in a newline on a
| unix-like or windows platform).
|
| Then consider the following program:
|
| PROGRAM eieio_stat
|   IMPLICIT NONE
|   INTEGER, PARAMETER :: unit = 10
|   INTEGER :: ios1, ios2, ios3
|   CHARACTER(100) :: buffer
|   !****
|   OPEN(UNIT=unit, FILE='file3.txt')  ! file 'xxx' has contents as above
|   READ (unit,"(A)",ADVANCE="NO",iostat=ios1) buffer
|   READ (unit,"(A)",ADVANCE="NO",iostat=ios2) buffer
|   READ (unit,"(A)",ADVANCE="NO",iostat=ios3) buffer
|   CLOSE(unit)
|   WRITE (*, "(3(A,'(',I0,')',:,', '))")  &
|       ioch(ios1), ios1, ioch(ios2), ios2, ioch(ios3), ios3
| CONTAINS
|   FUNCTION ioch(iostat)
|     USE, INTRINSIC :: ISO_FORTRAN_ENV, ONLY: IOSTAT_END, IOSTAT_EOR
|     INTEGER, INTENT(IN) :: iostat
|     CHARACTER(4) :: ioch
|     !****
|     IF (iostat == IOSTAT_END) THEN
|       ioch = 'IOSTAT_END'
|     ELSE IF (iostat == IOSTAT_EOR) THEN
|       ioch = 'IOSTAT_EOR'
|     ELSE IF (iostat < 0) THEN
|       ioch = 'Other -ve '
|     ELSE IF (iostat > 0) THEN
|       ioch = 'Positive  '
|     ELSE
|       ioch = 'Zero      '
|     END IF
|   END FUNCTION ioch
| END PROGRAM eieio_stat
|
| On Intel Fortran I get:
| IOSTAT_EOR(-2), IOSTAT_EOR(-2), IOSTAT_END(-1).
|
| On very recent gfortran 4.5.0 I get:
| IOSTAT_EOR(-2), Zero      (0), IOSTAT_END(-1).
|
| On g95 I get:
| IOSTAT_EOR(-2), Positive  (207), Positive  (207).
|
| These differences cause issues with the get procedure from
| iso_varying_string to read such "missing final newline" files and I've
| been trying to understand what the best/most robust way forward is
| (putting aside STREAM access for now).
|
| When attempting to rationalise the results, some of my conflicting
| thoughts have been:
|
| - Because the input file was not written by Fortran as a sequential
| formatted file (the way I am trying to read it) and misses the formally
| required end of record marker, then "anything goes" so I'm sunk and this
| whole post is irrelevant.
|
| - Intel's treatment is probably the most useful, the second EOR can be
| justified because the record did end (though it may have been missing a
| formal marker it's up to the processor to decide what the end of a
| record is?) and some useful stuff can be retrieved into buffer.
|
| - G95's treatment is not terribly useful, but still formally correct as
| the lack of an end of record marker is an error condition.  From that
| point on the unit is hosed.
|
| - gfortran's treatment is also useful, but a little inconsistent -
| something terminated the input into buffer, if it wasn't the end of
| record, end of file or some other error condition (ie. record
| incomplete), then what was it?
|
| By useful, I mean that I can get access to the text that is on the
| troublesome last line.

If you're using ADVANCE='NO', might be useful to
define BUFFER as having 1 character, not 100. 


0
Reply robin 2/3/2010 11:19:18 PM

On 4/02/2010 3:58 AM, Ron Shepard wrote:
> In article<1jdbapv.1ad8d151o18ny2N%nospam@see.signature>,
>   nospam@see.signature (Richard Maine) wrote:
>
>> For "improperly structured" files like the ones in question, you can
>> debate what you feel would be useful, and perhaps try to talk vendors
>> into doing it, but the standard doesn't offer any support.
>
> There are many unix utilities that will have problems with that last
> unterminated line, not just fortran compilers and fortran i/o
> libraries.  I would suggest shifting the emphasis back one step and
> try to fix the offending program that creates such files.
>
> $.02 -Ron Shepard

Thanks for the comments.

The files are generated by a text editor, which is operated by various 
human users.  Many times previously I've tried to fix these users, but 
with rather limited success.

I was reluctant to go down the formatted stream path as I wasn't sure 
what implications it would have for other IO code that reads from that 
unit.  Call me a chicken, or lazy, or both.  Also, some testing shows 
that some of the compilers I use have problems with it, though I'm also 
a common factor in that testing.

All I'm trying to do is to patch an iso_varying_string implementation 
(the one at www.fortran.com/iso_varying_string.f95) to be a bit more 
robust.

0
Reply Ian 2/4/2010 10:07:21 AM

Ian Harvey wrote:
> On 4/02/2010 3:58 AM, Ron Shepard wrote:
>> In article<1jdbapv.1ad8d151o18ny2N%nospam@see.signature>,
>>   nospam@see.signature (Richard Maine) wrote:
>>
>>> For "improperly structured" files like the ones in question, you can
>>> debate what you feel would be useful, and perhaps try to talk vendors
>>> into doing it, but the standard doesn't offer any support.
>>
>> There are many unix utilities that will have problems with that last
>> unterminated line, not just fortran compilers and fortran i/o
>> libraries.  I would suggest shifting the emphasis back one step and
>> try to fix the offending program that creates such files.
>>
>> $.02 -Ron Shepard
> 
> Thanks for the comments.
> 
> The files are generated by a text editor, which is operated by various 
> human users.  Many times previously I've tried to fix these users, but 
> with rather limited success.
> 
> I was reluctant to go down the formatted stream path as I wasn't sure 
> what implications it would have for other IO code that reads from that 
> unit.  Call me a chicken, or lazy, or both.  Also, some testing shows 
> that some of the compilers I use have problems with it, though I'm also 
> a common factor in that testing.
> 
> All I'm trying to do is to patch an iso_varying_string implementation 
> (the one at www.fortran.com/iso_varying_string.f95) to be a bit more 
> robust.
> 

I'm surprised that a text editor written in any language would save a 
file whose last line wasn't properly terminated.  That sort of thing is 
usually transparent to the user.

In any case, you might consider writing a utility program (in Fortran or 
C if that's easier) to fix the file.  Then your main Fortran application 
wouldn't have to worry about it.

Louis
0
Reply Louis 2/4/2010 12:15:17 PM

Louis Krupp <lkrupp_nospam@indra.com.invalid> wrote:
(snip)
 
> I'm surprised that a text editor written in any language would save a 
> file whose last line wasn't properly terminated.  That sort of thing is 
> usually transparent to the user.

For a "text editor" for editing plain ASCII files, yes.

It seems, though, that the convention for prose files is to use
the record delimiter character as a paragraph break, and not to store
one at the end of lines.  That allows for line breaks to flow
as the width changes.   In that case it seems usual not to supply
a paragraph break at the end of the file.  

Maybe there is an option to turn of this feature.

-- glen
0
Reply glen 2/4/2010 12:31:46 PM

On 3 feb, 09:14, Ian Harvey <ian_har...@bigpond.com> wrote:
> Consider a file, that is not of fortran I/O origin, that does not have
> an end of record maker prior to the end of file:
>
> Line 1<eor>
> Line 2<eof>
>
> where <eor> is the end of record marker, that also happens to be the
> line terminator for your platform of choice, and <eof> just means end of
> file (which may or may not involve some sort of explicit marker).
>
> (Basically - I've got a text file that doesn't end in a newline on a
> unix-like or windows platform).
>
> Then consider the following program:
>
> PROGRAM eieio_stat
> =A0 =A0IMPLICIT NONE
> =A0 =A0INTEGER, PARAMETER :: unit =3D 10
> =A0 =A0INTEGER :: ios1, ios2, ios3
> =A0 =A0CHARACTER(100) :: buffer
> =A0 =A0!****
> =A0 =A0OPEN(UNIT=3Dunit, FILE=3D'file3.txt') =A0! file 'xxx' has contents=
 as above
> =A0 =A0READ (unit,"(A)",ADVANCE=3D"NO",iostat=3Dios1) buffer
> =A0 =A0READ (unit,"(A)",ADVANCE=3D"NO",iostat=3Dios2) buffer
> =A0 =A0READ (unit,"(A)",ADVANCE=3D"NO",iostat=3Dios3) buffer
> =A0 =A0CLOSE(unit)
> =A0 =A0WRITE (*, "(3(A,'(',I0,')',:,', '))") =A0&
> =A0 =A0 =A0 =A0ioch(ios1), ios1, ioch(ios2), ios2, ioch(ios3), ios3
> CONTAINS
> =A0 =A0FUNCTION ioch(iostat)
> =A0 =A0 =A0USE, INTRINSIC :: ISO_FORTRAN_ENV, ONLY: IOSTAT_END, IOSTAT_EO=
R
> =A0 =A0 =A0INTEGER, INTENT(IN) :: iostat
> =A0 =A0 =A0CHARACTER(4) :: ioch
> =A0 =A0 =A0!****
> =A0 =A0 =A0IF (iostat =3D=3D IOSTAT_END) THEN
> =A0 =A0 =A0 =A0ioch =3D 'IOSTAT_END'
> =A0 =A0 =A0ELSE IF (iostat =3D=3D IOSTAT_EOR) THEN
> =A0 =A0 =A0 =A0ioch =3D 'IOSTAT_EOR'
> =A0 =A0 =A0ELSE IF (iostat < 0) THEN
> =A0 =A0 =A0 =A0ioch =3D 'Other -ve '
> =A0 =A0 =A0ELSE IF (iostat > 0) THEN
> =A0 =A0 =A0 =A0ioch =3D 'Positive =A0'
> =A0 =A0 =A0ELSE
> =A0 =A0 =A0 =A0ioch =3D 'Zero =A0 =A0 =A0'
> =A0 =A0 =A0END IF
> =A0 =A0END FUNCTION ioch
> END PROGRAM eieio_stat
>
> On Intel Fortran I get:
> IOSTAT_EOR(-2), IOSTAT_EOR(-2), IOSTAT_END(-1).
>
> On very recent gfortran 4.5.0 I get:
> IOSTAT_EOR(-2), Zero =A0 =A0 =A0(0), IOSTAT_END(-1).
>
> On g95 I get:
> IOSTAT_EOR(-2), Positive =A0(207), Positive =A0(207).
>
> These differences cause issues with the get procedure from
> iso_varying_string to read such "missing final newline" files and I've
> been trying to understand what the best/most robust way forward is
> (putting aside STREAM access for now).
>
> When attempting to rationalise the results, some of my conflicting
> thoughts have been:
>
> - Because the input file was not written by Fortran as a sequential
> formatted file (the way I am trying to read it) and misses the formally
> required end of record marker, then "anything goes" so I'm sunk and this
> whole post is irrelevant.
>
> - Intel's treatment is probably the most useful, the second EOR can be
> justified because the record did end (though it may have been missing a
> formal marker it's up to the processor to decide what the end of a
> record is?) and some useful stuff can be retrieved into buffer.
>
> - G95's treatment is not terribly useful, but still formally correct as
> the lack of an end of record marker is an error condition. =A0From that
> point on the unit is hosed.
>
> - gfortran's treatment is also useful, but a little inconsistent -
> something terminated the input into buffer, if it wasn't the end of
> record, end of file or some other error condition (ie. record
> incomplete), then what was it?
>
> By useful, I mean that I can get access to the text that is on the
> troublesome last line.
>
> Thoughts?
>
> Thanks,
>
> IanH

How about a program like this?

! incomplete.f90 --
!     Read an incomplete file
!
program incomplete

    implicit none

    integer           :: i, k, ierr
    character(len=3D80) :: line
    character(len=3D1)  :: char

    open( 10, file =3D 'incomplete.inp' )
    line =3D 'Some line of text, written without an EOL character'
    write( 10, '(a)' ) 'First line'
    write( 10, '(a)', advance =3D 'no' ) trim(line)
    close( 10 )
!
! Try read the file
!
    open( 10, file =3D 'incomplete.inp' )
    line =3D ' '
    k    =3D 0
    do i =3D 1,100
        k =3D k + 1
        read( 10, '(a)', iostat =3D ierr, advance =3D 'no' ) char
        if ( ierr =3D=3D -2 ) then
            write(*,*) ierr, trim(line)
            read( 10, '(a)', iostat =3D ierr, advance =3D 'no' ) char
            k =3D 1
            line =3D ' '
        else if ( ierr =3D=3D -1 ) then
            write(*,*) ierr, trim(line)
            exit
        endif
        line(k:k) =3D char
    enddo
end program

The trick is to read the file character by character, so that
nothing gets lost.

It is annoying that one has to go through such lengths, but
that is the real world for you.

Regards,

Arjen
0
Reply Arjen 2/4/2010 1:37:56 PM

On 2010-02-04 06:07:21 -0400, Ian Harvey <ian_harvey@bigpond.com> said:

> On 4/02/2010 3:58 AM, Ron Shepard wrote:
>> In article<1jdbapv.1ad8d151o18ny2N%nospam@see.signature>,
>>   nospam@see.signature (Richard Maine) wrote:
>> 
>>> For "improperly structured" files like the ones in question, you can
>>> debate what you feel would be useful, and perhaps try to talk vendors
>>> into doing it, but the standard doesn't offer any support.
>> 
>> There are many unix utilities that will have problems with that last
>> unterminated line, not just fortran compilers and fortran i/o
>> libraries.  I would suggest shifting the emphasis back one step and
>> try to fix the offending program that creates such files.
>> 
>> $.02 -Ron Shepard
> 
> Thanks for the comments.
> 
> The files are generated by a text editor, which is operated by various 
> human users.  Many times previously I've tried to fix these users, but 
> with rather limited success.

I seem to recall seeing an option deep in the "more obscure options 
section" of some
editors that they would fix the mising final line end. All the editors 
I have experience
with will open multiple files easily and then equally easily close the 
files. This
is just a special case of finding some utility that will copy the files 
and ensure
that the output is well formed. The couple editors I use all have 
options to use
any of the three major line end conventions. I would expect that 
finding a programmer's
text editor that can both convert and fix line ends is mostly a 
willingness to read the
fine print on options for a couple of them. Being allowed to buy the 
software and
use it may be a different class of problem. :-( This is a case where 
the adjective
in front of editor matters. Programmer's text editors are not word 
processors as
they offer zero formatting and will only change the display font size 
for the whole
file as an indulgence for the users who want easier to read screens. 
Such editors are
sold on their features which help the manual production of HTML and 
such so features
like line end control may be harder to detemine.

> I was reluctant to go down the formatted stream path as I wasn't sure 
> what implications it would have for other IO code that reads from that 
> unit.  Call me a chicken, or lazy, or both.  Also, some testing shows 
> that some of the compilers I use have problems with it, though I'm also 
> a common factor in that testing.
> 
> All I'm trying to do is to patch an iso_varying_string implementation 
> (the one at www.fortran.com/iso_varying_string.f95) to be a bit more 
> robust.


0
Reply Gordon 2/4/2010 2:04:09 PM

glen herrmannsfeldt <gah@ugcs.caltech.edu> wrote:

> Louis Krupp <lkrupp_nospam@indra.com.invalid> wrote:
> (snip)
>  
> > I'm surprised that a text editor written in any language would save a
> > file whose last line wasn't properly terminated.  That sort of thing is
> > usually transparent to the user.
> 
> For a "text editor" for editing plain ASCII files, yes.

I've seen multiple editors for "plain ASCII files" that have the
mentioned problem. Emacs comes to mind. At least in the case of Emacs,
there is an option to automatically add a final terminator as needed,
but that option at least used to default to being off. I recall it
because it was part of my usual customization of Emacs to turn it on.

-- 
Richard Maine                    | Good judgment comes from experience;
email: last name at domain . net | experience comes from bad judgment.
domain: summertriangle           |  -- Mark Twain
0
Reply nospam 2/4/2010 4:44:59 PM

In article <1jdds6z.bkp2ud6tbooyN%nospam@see.signature>,
 nospam@see.signature (Richard Maine) wrote:

> glen herrmannsfeldt <gah@ugcs.caltech.edu> wrote:
> 
> > Louis Krupp <lkrupp_nospam@indra.com.invalid> wrote:
> > (snip)
> >  
> > > I'm surprised that a text editor written in any language would save a
> > > file whose last line wasn't properly terminated.  That sort of thing is
> > > usually transparent to the user.
> > 
> > For a "text editor" for editing plain ASCII files, yes.
> 
> I've seen multiple editors for "plain ASCII files" that have the
> mentioned problem. Emacs comes to mind. At least in the case of Emacs,
> there is an option to automatically add a final terminator as needed,
> but that option at least used to default to being off. I recall it
> because it was part of my usual customization of Emacs to turn it on.

I've also used editors in the past that would print a warning 
message whenever a file was written with an unterminated line.  I 
thought vi did this, but when I checked just a minute ago, it wrote 
the file with no such warning.  So maybe some option needs to be set 
or something.

I normally use emacs, but this error has bitten me so often in the 
distant past that I always make sure the last line is correct just 
by instinct, I don't even think about it any more.  I'll look for 
the option to add the termination automatically, that looks like a 
good idea.  I can't think of any situation where I would want to 
edit a file without the termination.

$.02 -Ron Shepard
0
Reply Ron 2/4/2010 5:41:00 PM

Richard Maine wrote:
> glen herrmannsfeldt <gah@ugcs.caltech.edu> wrote:
> 
>> Louis Krupp <lkrupp_nospam@indra.com.invalid> wrote:
>> (snip)
>>  
>>> I'm surprised that a text editor written in any language would save a
>>> file whose last line wasn't properly terminated.  That sort of thing is
>>> usually transparent to the user.
>> For a "text editor" for editing plain ASCII files, yes.
> 
> I've seen multiple editors for "plain ASCII files" that have the
> mentioned problem. Emacs comes to mind. At least in the case of Emacs,
> there is an option to automatically add a final terminator as needed,
> but that option at least used to default to being off. I recall it
> because it was part of my usual customization of Emacs to turn it on.
> 

You're right.  I just tried emacs and resisted the temptation to hit 
return at the end of the last line like I always do without thinking.

For the OP, I would try (in this order):

1.  Set up a .rc file or something with an option to make the editor add 
the trailing newline without the user having to worry about it.

2.  Wrap the application in a shell script or something that first runs 
a utility to add the newline if necessary.  You could probably do this 
in fifteen lines of C (and I don't think anyone will argue that, 
whatever your opinion is of C, this is a good use for it).

3.  Change the application to accommodate a truncated input file.

Louis
0
Reply Louis 2/4/2010 5:46:11 PM

Ron Shepard <ron-shepard@NOSPAM.comcast.net> wrote:

> In article <1jdds6z.bkp2ud6tbooyN%nospam@see.signature>,
>  nospam@see.signature (Richard Maine) wrote:

> > At least in the case of Emacs,
> > there is an option to automatically add a final terminator as needed,
> > but that option at least used to default to being off. I recall it
> > because it was part of my usual customization of Emacs to turn it on.
....
> I normally use emacs, ....  I'll look for 
> the option to add the termination automatically,

Extracted from my personal customization files:

  (setq require-final-newline t)  ; Automatically add newline at end of
file.

It is possible that might be specific to xemacs, which was the
particular emacs denomination where I worshipped at the time.

-- 
Richard Maine                    | Good judgment comes from experience;
email: last name at domain . net | experience comes from bad judgment.
domain: summertriangle           |  -- Mark Twain
0
Reply nospam 2/4/2010 5:58:27 PM

On 5/02/2010 12:37 AM, Arjen Markus wrote:
....
> How about a program like this?
>
> ! incomplete.f90 --
> !     Read an incomplete file
> !
> program incomplete
>
>      implicit none
>
>      integer           :: i, k, ierr
>      character(len=80) :: line
>      character(len=1)  :: char
>
>      open( 10, file = 'incomplete.inp' )
>      line = 'Some line of text, written without an EOL character'
>      write( 10, '(a)' ) 'First line'
>      write( 10, '(a)', advance = 'no' ) trim(line)
>      close( 10 )
> !
> ! Try read the file
> !
>      open( 10, file = 'incomplete.inp' )
>      line = ' '
>      k    = 0
>      do i = 1,100
>          k = k + 1
>          read( 10, '(a)', iostat = ierr, advance = 'no' ) char
>          if ( ierr == -2 ) then
>              write(*,*) ierr, trim(line)
>              read( 10, '(a)', iostat = ierr, advance = 'no' ) char
>              k = 1
>              line = ' '
>          else if ( ierr == -1 ) then
>              write(*,*) ierr, trim(line)
>              exit
>          endif
>          line(k:k) = char
>      enddo
> end program
>
> The trick is to read the file character by character, so that
> nothing gets lost.
>
> It is annoying that one has to go through such lengths, but
> that is the real world for you.
>
> Regards,
>
> Arjen

Thanks for the suggestion.

Curiously, on the few compilers I was testing when I was trying to set 
up test cases, I couldn't get them to generate an appropriately dodgy 
input file.  A record terminator (newline) got added (using the same 
code sequence as you have above) when the file was closed, even in 
stream access mode.  I had to use a text editor to make a broken one.

I think this char by char stuff is where I'll end up going.
0
Reply Ian 2/4/2010 9:01:20 PM

In article <1jddvgk.aukxzunu18wwN%nospam@see.signature>,
 nospam@see.signature (Richard Maine) wrote:

> Ron Shepard <ron-shepard@NOSPAM.comcast.net> wrote:
> 
> > In article <1jdds6z.bkp2ud6tbooyN%nospam@see.signature>,
> >  nospam@see.signature (Richard Maine) wrote:
> 
> > > At least in the case of Emacs,
> > > there is an option to automatically add a final terminator as needed,
> > > but that option at least used to default to being off. I recall it
> > > because it was part of my usual customization of Emacs to turn it on.
> ...
> > I normally use emacs, ....  I'll look for 
> > the option to add the termination automatically,
> 
> Extracted from my personal customization files:
> 
>   (setq require-final-newline t)  ; Automatically add newline at end of
> file.
> 
> It is possible that might be specific to xemacs, which was the
> particular emacs denomination where I worshipped at the time.

Thanks.  In the standard emacs version in MacOSX (22.1.1), that variable 
is set to t when in f90 and text modes, but not necessarily in other 
file modes.

$.02 -Ron Shepard
0
Reply Ron 2/4/2010 9:06:18 PM

On 4 feb, 18:41, Ron Shepard <ron-shep...@NOSPAM.comcast.net> wrote:
> In article <1jdds6z.bkp2ud6tbooyN%nos...@see.signature>,
> =A0nos...@see.signature (Richard Maine) wrote:
>
>
>
>
>
> > glen herrmannsfeldt <g...@ugcs.caltech.edu> wrote:
>
> > > Louis Krupp <lkrupp_nos...@indra.com.invalid> wrote:
> > > (snip)
>
> > > > I'm surprised that a text editor written in any language would save=
 a
> > > > file whose last line wasn't properly terminated. =A0That sort of th=
ing is
> > > > usually transparent to the user.
>
> > > For a "text editor" for editing plain ASCII files, yes.
>
> > I've seen multiple editors for "plain ASCII files" that have the
> > mentioned problem. Emacs comes to mind. At least in the case of Emacs,
> > there is an option to automatically add a final terminator as needed,
> > but that option at least used to default to being off. I recall it
> > because it was part of my usual customization of Emacs to turn it on.
>
> I've also used editors in the past that would print a warning
> message whenever a file was written with an unterminated line. =A0I
> thought vi did this, but when I checked just a minute ago, it wrote
> the file with no such warning. =A0So maybe some option needs to be set
> or something.
>
> I normally use emacs, but this error has bitten me so often in the
> distant past that I always make sure the last line is correct just
> by instinct, I don't even think about it any more. =A0I'll look for
> the option to add the termination automatically, that looks like a
> good idea. =A0I can't think of any situation where I would want to
> edit a file without the termination.
>
> $.02 -Ron Shepard- Tekst uit oorspronkelijk bericht niet weergeven -
>
> - Tekst uit oorspronkelijk bericht weergeven -

I remember this as a message from vi when it _reads_ the file:
Something like "incomplete last line"

Regards,

Arjen
0
Reply Arjen 2/5/2010 7:49:30 AM

On 4 feb, 22:01, Ian Harvey <ian_har...@bigpond.com> wrote:
> On 5/02/2010 12:37 AM, Arjen Markus wrote:
> ...
>
>
>
>
>
> > How about a program like this?
>
> > ! incomplete.f90 --
> > ! =A0 =A0 Read an incomplete file
> > !
> > program incomplete
>
> > =A0 =A0 =A0implicit none
>
> > =A0 =A0 =A0integer =A0 =A0 =A0 =A0 =A0 :: i, k, ierr
> > =A0 =A0 =A0character(len=3D80) :: line
> > =A0 =A0 =A0character(len=3D1) =A0:: char
>
> > =A0 =A0 =A0open( 10, file =3D 'incomplete.inp' )
> > =A0 =A0 =A0line =3D 'Some line of text, written without an EOL characte=
r'
> > =A0 =A0 =A0write( 10, '(a)' ) 'First line'
> > =A0 =A0 =A0write( 10, '(a)', advance =3D 'no' ) trim(line)
> > =A0 =A0 =A0close( 10 )
> > !
> > ! Try read the file
> > !
> > =A0 =A0 =A0open( 10, file =3D 'incomplete.inp' )
> > =A0 =A0 =A0line =3D ' '
> > =A0 =A0 =A0k =A0 =A0=3D 0
> > =A0 =A0 =A0do i =3D 1,100
> > =A0 =A0 =A0 =A0 =A0k =3D k + 1
> > =A0 =A0 =A0 =A0 =A0read( 10, '(a)', iostat =3D ierr, advance =3D 'no' )=
 char
> > =A0 =A0 =A0 =A0 =A0if ( ierr =3D=3D -2 ) then
> > =A0 =A0 =A0 =A0 =A0 =A0 =A0write(*,*) ierr, trim(line)
> > =A0 =A0 =A0 =A0 =A0 =A0 =A0read( 10, '(a)', iostat =3D ierr, advance =
=3D 'no' ) char
> > =A0 =A0 =A0 =A0 =A0 =A0 =A0k =3D 1
> > =A0 =A0 =A0 =A0 =A0 =A0 =A0line =3D ' '
> > =A0 =A0 =A0 =A0 =A0else if ( ierr =3D=3D -1 ) then
> > =A0 =A0 =A0 =A0 =A0 =A0 =A0write(*,*) ierr, trim(line)
> > =A0 =A0 =A0 =A0 =A0 =A0 =A0exit
> > =A0 =A0 =A0 =A0 =A0endif
> > =A0 =A0 =A0 =A0 =A0line(k:k) =3D char
> > =A0 =A0 =A0enddo
> > end program
>
> > The trick is to read the file character by character, so that
> > nothing gets lost.
>
> > It is annoying that one has to go through such lengths, but
> > that is the real world for you.
>
> > Regards,
>
> > Arjen
>
> Thanks for the suggestion.
>
> Curiously, on the few compilers I was testing when I was trying to set
> up test cases, I couldn't get them to generate an appropriately dodgy
> input file. =A0A record terminator (newline) got added (using the same
> code sequence as you have above) when the file was closed, even in
> stream access mode. =A0I had to use a text editor to make a broken one.
>
> I think this char by char stuff is where I'll end up going.- Tekst uit oo=
rspronkelijk bericht niet weergeven -
>
> - Tekst uit oorspronkelijk bericht weergeven -

Really? I used gfortran - an old one it turns out: 4.1.x, maybe that
has changed since then.

Regards,

Arjen

0
Reply Arjen 2/5/2010 7:51:51 AM

Arjen Markus <arjen.markus895@gmail.com> wrote:
 
> I remember this as a message from vi when it _reads_ the file:
> Something like "incomplete last line"

It does have that message.  As far as I know, though, it won't
create files that way.  If you edit one and save it then the
last line gets its "\n" added.

I just tested it with vim to be sure.  Maybe not all vi though.

I haven't used emacs much, but it might be that it can edit
non-text files where the ability to create files without line
terminators was needed.  Not so convenient for a default, but
sometimes useful.

-- glen
0
Reply glen 2/5/2010 10:41:03 AM

Ian Harvey wrote:
> Consider a file, that is not of fortran I/O origin, that does not have 
> an end of record maker prior to the end of file:
> 
> Line 1<eor>
> Line 2<eof>
> 
> where <eor> is the end of record marker, that also happens to be the 
> line terminator for your platform of choice, and <eof> just means end of 
> file (which may or may not involve some sort of explicit marker).
> 
--- snip ---
> - gfortran's treatment is also useful, but a little inconsistent - 
> something terminated the input into buffer, if it wasn't the end of 
> record, end of file or some other error condition (ie. record 
> incomplete), then what was it?
> 
> By useful, I mean that I can get access to the text that is on the 
> troublesome last line.
> 
> Thoughts?

Not of Fortran origin is not relevant. For those who care, the example file can 
be created using Fortran with either STREAM access or using the $ edit 
descriptor.  Let's move on to the important part.

Look at the F2003 standard.  Seems pretty clear and well defined to me what this 
should do.  I have submitted a patch to gfortran to fix this.  With PAD="yes", 
which is the default if not specified, the EOR or EOF conditions should be set 
and the list item result is not "undefined". With PAD="no", the results are 
"undefined" and GFortran chooses to leave "buffer" untouched.  I appreciate your 
presenting this test case.

9.10.3 (1)

"If the pad mode has the value YES, the record is padded with blanks to satisfy 
the input list item (9.5.3.4.2) and corresponding data edit descriptor that 
requires more characters than the record contains. If the pad mode has the value 
NO, the input list item becomes undefined."

9.5.4.3.2

"During nonadvancing input when the pad mode has the value NO, an end-of-record 
condition (9.10) occurs if the input list and format specification require more 
characters from the record than the record contains, and the record is complete 
(9.2.2.3). If the record is incomplete, an end-of-file condition occurs instead 
of an end-of-record condition.

During nonadvancing input when the pad mode has the value YES, blank characters 
are supplied by the processor if an input item and its corresponding data edit 
descriptor require more characters from the record than the record contains. If 
the record is incomplete, an end-of-file condition occurs; otherwise an 
end-of-record condition occurs."

Best regards,

Jerry
0
Reply jvdelisle (55) 2/6/2010 6:20:18 AM

Jerry DeLisle <jvdelisle@verizon.net> wrote:

> For those who care, the example file can 
> be created using Fortran with either STREAM access or using the $ edit
> descriptor.  Let's move on to the important part.

The $ edit descriptor is nonstandard.

> Look at the F2003 standard.

Yes. But note that all the relevant material is new to f2003 and, in
particular, was introduced in conjunction with stream access. Note that
the definition of an incomplete record is in the section on stream
access (and that's the only way you can create such a thing with
standard f2003). Prior to the introduction of stream access, the Fortran
standard had no concept of an incomplete record.

Thus for f95, there is no standard-specified answer. For f2003, well...
the OP said "putting aside STREAM access for now." I'm not clear on why
he put stream access aside, as I think that the best answer. The ones
you cited are basically a side effect of trying to integrate stream
access with record concepts.

-- 
Richard Maine                    | Good judgment comes from experience;
email: last name at domain . net | experience comes from bad judgment.
domain: summertriangle           |  -- Mark Twain
0
Reply nospam47 (9742) 2/6/2010 6:41:15 AM

Richard Maine wrote:
> Jerry DeLisle <jvdelisle@verizon.net> wrote:
> 
>> For those who care, the example file can 
>> be created using Fortran with either STREAM access or using the $ edit
>> descriptor.  Let's move on to the important part.
> 
> The $ edit descriptor is nonstandard.
> 
>> Look at the F2003 standard.
> 
> Yes. But note that all the relevant material is new to f2003 and, in
> particular, was introduced in conjunction with stream access. Note that
> the definition of an incomplete record is in the section on stream
> access (and that's the only way you can create such a thing with
> standard f2003). Prior to the introduction of stream access, the Fortran
> standard had no concept of an incomplete record.
> 
> Thus for f95, there is no standard-specified answer. For f2003, well...
> the OP said "putting aside STREAM access for now." I'm not clear on why
> he put stream access aside, as I think that the best answer. The ones
> you cited are basically a side effect of trying to integrate stream
> access with record concepts.
> 
All true.

We are moving toward F2003 and even F2008. If the feature is there and it is 
Standard complying, people should not hesitate to use it.

Putting semantics aside, a record that has a missing <EOR> seems very much 
incomplete to me and I have no problem taking the broader interpretation.  Side 
effect or not, it makes good sense to me to do what it says.  The read is 
hitting the end of the file.  I would suggest that if further clarification is 
needed, then the standards committee should do so. (And the engineer said as he 
approached the girl half the distance ... close enough! ... and he kissed her  ;)  )

Best Regards,

Jerry

0
Reply jvdelisle (55) 2/6/2010 7:10:01 AM

20 Replies
264 Views

(page loaded in 0.214 seconds)


Reply: