Can anyone help me with a sed command to add a newline to the last
line of STDIN if it does not end in \n?

e.g.:
cat file_w_no_eol | sed '...' > file_w_eol

But it would need to be conditional; a file with 2 newlines on the end
wouldn't work.

Thanks,
Phil

$ uname -X
System = SunOS
Node = ...
Release = 5.9
KernelID = Generic_118558-20
Machine = sun4us
BusType = <unknown>
Serial = <unknown>
Users = <unknown>
OEM# = 0
Origin# = 1
NumCPU = 48

In article <1174340070.109459.27060@e1g2000hsg.googlegroups.com>,
Phil Lawrence <prlawrence@gmail.com> wrote:
>Can anyone help me with a sed command to add a newline to the last
>line of STDIN if it does not end in \n?
>
>e.g.:
>cat file_w_no_eol | sed '...' > file_w_eol
>
>But it would need to be conditional; a file with 2 newlines on the end
>wouldn't work.

sounds tricky in sed.  is this an ascii file?  binary?
if it's ascii, a flex/lex program might be more straightforward.
j.

>
>Thanks,
>Phil
>
>$ uname -X
>System = SunOS
>Node = ...
>Release = 5.9
>KernelID = Generic_118558-20
>Machine = sun4us
>BusType = <unknown>
>Serial = <unknown>
>Users = <unknown>
>OEM# = 0
>Origin# = 1
>NumCPU = 48
>


-- 
Jay Scott		512-835-3553		gl@arlut.utexas.edu
Head of Sun Support, Sr. Operating Systems Specialist
Applied Research Labs, Computer Science Div.                   S224
University of Texas at Austin

On Mar 19, 4:34 pm, "Phil Lawrence" <prlawre...@gmail.com> wrote:
> Can anyone help me with a sed command to add a newline to the last
> line of STDIN if it does not end in \n?
>
> e.g.:
> cat file_w_no_eol | sed '...' > file_w_eol
>
> But it would need to be conditional; a file with 2 newlines on the end
> wouldn't work.
>
> Thanks,
> Phil
>
> $ uname -X
> System = SunOS
> Node = ...
> Release = 5.9
> KernelID = Generic_118558-20
> Machine = sun4us
> BusType = <unknown>
> Serial = <unknown>
> Users = <unknown>
> OEM# = 0
> Origin# = 1
> NumCPU = 48

Phil:

Do you have to solve this problem exclusively with sed?

When I read your question I remembered that 'wc' and 'awk' count lines
in a different way when there is no EOL at the end of the file.

If you use 'wc' in a non-standard file:

% wc -l filename.txt

it will say that has 1 line less than if you use 'awk':

% awk {print NR} filename.txt | tail -1

OTOH, if the file has the correct number of EOLs, both programs will
report the same number of lines.

-Ramon

In article <1174342627.266691.23000@l77g2000hsb.googlegroups.com>,
Ramon F Herrera <ramon@conexus.net> wrote:
>On Mar 19, 4:34 pm, "Phil Lawrence" <prlawre...@gmail.com> wrote:
>> Can anyone help me with a sed command to add a newline to the last
>> line of STDIN if it does not end in \n?
>>


[snip]

here's a cheap flex program which works for me.
FWIW.  maybe you can do it w/ sed, wc, and awk,
like the previous poster said, which would be
easier to document.



81 > more lll.l
%{
#include <stdlib.h>
int wasnl;
int snooble(void);
%}
%%

..+      {ECHO;wasnl=0;}
\n      {ECHO;wasnl=1;}
<<EOF>> {snooble();}

%%
int snooble(void)
{
if( wasnl == 0 )
        printf("\n");
exit(0);
return 0;
}


-- 
Jay Scott		512-835-3553		gl@arlut.utexas.edu
Head of Sun Support, Sr. Operating Systems Specialist
Applied Research Labs, Computer Science Div.                   S224
University of Texas at Austin

On Mar 19, 5:17 pm, "Ramon F Herrera" <r...@conexus.net> wrote:
> On Mar 19, 4:34 pm, "Phil Lawrence" <prlawre...@gmail.com> wrote:
>
> > Can anyone help me with a sed command to add a newline to the last
> > line of STDIN if it does not end in \n?

> Do you have to solve this problem exclusively with sed?
>
> When I read your question I remembered that 'wc' and 'awk' count lines
> in a different way when there is no EOL at the end of the file.
> ...
> OTOH, if the file has the correct number of EOLs, both programs will
> report the same number of lines.

Good thought!  I'll think about how to use this test in the pipe...

Thank you,
Phil

On Mar 19, 5:21 pm, g...@csdsun1.arlut.utexas.edu (Jay G. Scott)
wrote:
> In article <1174342627.266691.23...@l77g2000hsb.googlegroups.com>,
> Ramon F Herrera <r...@conexus.net> wrote:
>
> >On Mar 19, 4:34 pm, "Phil Lawrence" <prlawre...@gmail.com> wrote:
> >> Can anyone help me with a sed command to add a newline to the last
> >> line of STDIN if it does not end in \n?
>
> [snip]
>
> here's a cheap flex program which works for me.
> FWIW.  maybe you can do it w/ sed, wc, and awk,
> like the previous poster said, which would be
> easier to document.
>
> [snip]

Wow.  You happen to be the first person to ever mention flex to me.
Never heard of it before.  Have *heard* of lex...  looks pretty
powerful.

Thanks,
Phil

Another idea:

On Mar 19, 2:34 pm, "Phil Lawrence" <prlawre...@gmail.com> wrote:
> Can anyone help me with a sed command to add a newline to the last
> line of STDIN if it does not end in \n?
>
> e.g.:
> cat file_w_no_eol | sed '...' > file_w_eol
>
> But it would need to be conditional; a file with 2 newlines on the end
> wouldn't work.
>
> Thanks,
> Phil

Is there data on each and every line?

Try this:

Preparation:
echo "" > eol_only

Operation:
cat file_eol_maybe eol_only | awk 'NF > 0' > correct_file

If there's data on the line, awk's "NF" will be greater than 0.  The
line with only an EOL will have "NF==0", and awk won't print it.

Altho, personally, I like Jay G. Scott's "flex" solution, except for
the need to acquire "flex".  This is exactly the kind of problem I'd
use "flex" to solve.

Best of luck!

In article <1174352803.560378.216070@b75g2000hsg.googlegroups.com>,
ThanksButNo <no.no.thanks@gmail.com> wrote:
>Another idea:
>
>On Mar 19, 2:34 pm, "Phil Lawrence" <prlawre...@gmail.com> wrote:
>> Can anyone help me with a sed command to add a newline to the last
>> line of STDIN if it does not end in \n?
>>
>> e.g.:
>> cat file_w_no_eol | sed '...' > file_w_eol
>>
>> But it would need to be conditional; a file with 2 newlines on the end
>> wouldn't work.
>>
>> Thanks,
>> Phil
>
>Is there data on each and every line?
>
>Try this:
>
>Preparation:
>echo "" > eol_only
>
>Operation:
>cat file_eol_maybe eol_only | awk 'NF > 0' > correct_file
>
>If there's data on the line, awk's "NF" will be greater than 0.  The
>line with only an EOL will have "NF==0", and awk won't print it.
>
>Altho, personally, I like Jay G. Scott's "flex" solution, except for
>the need to acquire "flex".  This is exactly the kind of problem I'd
>use "flex" to solve.
>
>Best of luck!
>
>
>

just occurs to me:
use echo to tack on a newline unconditionally.
then write a sed script which deletes the last line
if it is blank.  you may have to dig up a sed manual
to see how to do that, but i bet it can be done.

j.


-- 
Jay Scott		512-835-3553		gl@arlut.utexas.edu
Head of Sun Support, Sr. Operating Systems Specialist
Applied Research Labs, Computer Science Div.                   S224
University of Texas at Austin

In article <etotd8$1t$1@ns3.arlut.utexas.edu>,
	gl@csdsun1.arlut.utexas.edu (Jay G. Scott) writes:
> In article <1174352803.560378.216070@b75g2000hsg.googlegroups.com>,
> ThanksButNo <no.no.thanks@gmail.com> wrote:
>>Another idea:
>>
>>On Mar 19, 2:34 pm, "Phil Lawrence" <prlawre...@gmail.com> wrote:
>>> Can anyone help me with a sed command to add a newline to the last
>>> line of STDIN if it does not end in \n?
>>>
>>> e.g.:
>>> cat file_w_no_eol | sed '...' > file_w_eol
>>>
>>> But it would need to be conditional; a file with 2 newlines on the end
>>> wouldn't work.
>>>
>>> Thanks,
>>> Phil
>>
>>Is there data on each and every line?
>>
>>Try this:
>>
>>Preparation:
>>echo "" > eol_only
>>
>>Operation:
>>cat file_eol_maybe eol_only | awk 'NF > 0' > correct_file
>>
>>If there's data on the line, awk's "NF" will be greater than 0.  The
>>line with only an EOL will have "NF==0", and awk won't print it.
>>
>>Altho, personally, I like Jay G. Scott's "flex" solution, except for
>>the need to acquire "flex".  This is exactly the kind of problem I'd
>>use "flex" to solve.
>>
>>Best of luck!
>>
>>
>>
> 
> just occurs to me:
> use echo to tack on a newline unconditionally.
> then write a sed script which deletes the last line
> if it is blank.  you may have to dig up a sed manual
> to see how to do that, but i bet it can be done.
> 
> j.
> 

I'd tried to come up with a way to do this really efficiently, for
regular files (rather than possibly unseekable data streams).  I think
something along the lines of (ignoring zero size files and error checking
here for the sake of simplicity):

    char c;
    ...
    lseek(fd,(off_t) -1, SEEK_END);
    read(fd,&c,1);
    if (c!='\n')
         write(fd,"\n",1);

would be fine for one-byte-per-character locales (like C, *.ISO8859-*, etc).
If I understand correctly, UTF-8 multi-byte character representations
(those outside the 0-127 range) always have the high order bit on in each
byte, so that shouldn't be misled to think there was a newline present
when there really wasn't.  I mean, it couldn't work on an encoding
that didn't have the property of never having a '\n' that stood for
something other than itself, so it wouldn't work right on
UTF-16/UTF-32/SJIS, etc.  But AFAIK, none of those actually have locales
on Solaris (although they have iconv support), so I would think that
aside from needing a seekable fd, that would be as "good" as using some
locale-aware tool, and far more efficient.  Or did I miss something?


-- 
eMail:              mailto:rlhamil@smart.net
Home page:          http://www.smart.net/~rlhamil
MySpace:            http://www.myspace.com/yesterdays_geek
AIM, Yahoo, etc:    ask

On 19 Mar, 23:21, g...@csdsun1.arlut.utexas.edu (Jay G. Scott) wrote:
> In article <1174342627.266691.23...@l77g2000hsb.googlegroups.com>,
> Ramon F Herrera <r...@conexus.net> wrote:
>
> >On Mar 19, 4:34 pm, "Phil Lawrence" <prlawre...@gmail.com> wrote:
> >> Can anyone help me with a sed command to add a newline to the last
> >> line of STDIN if it does not end in \n?
>
> [snip]
>
> here's a cheap flex program which works for me.
> FWIW.  maybe you can do it w/ sed, wc, and awk,
> like the previous poster said, which would be
> easier to document.
>
> 81 > more lll.l
> %{
> #include <stdlib.h>
> int wasnl;
> int snooble(void);
> %}
> %%
>
> .+      {ECHO;wasnl=0;}
> \n      {ECHO;wasnl=1;}
> <<EOF>> {snooble();}
>
> %%
> int snooble(void)
> {
> if( wasnl == 0 )
>         printf("\n");
> exit(0);
> return 0;
>
> }

If you're going to use flex you may as well use C.

#include <stdio.h>

int main(void) {
    int a , b=EOF ;

    while ( (a=getchar()) != EOF ) {
        putchar(a) ;
        b=a ;
    }
    if (b != '\n' && b != EOF) putchar('\n') ;
    return 0 ;
}

<Not tested>