Large file problems - block pattern

  • Follow


Hi,  I have 2 questions

1/ The command below prints every matched pattern blocks w/o blank RS
record separator.
How can I print out only FIRST and LAST pattern blocks with blank RS.

cat infile | awk ' BEGIN { RS = "" } /REBUIL/ { print } '

2/ The infile is 260573bytes, but awk only can read 3000bytes .
Is there any suggestion on file size limitation ?

cat infile | awk ' BEGIN { RS = "" } /REBUIL/ { print } '
awk: Input line Controller At M/C1:
 cannot be longer than 3,000 bytes.
 The input line number is 169.
 The source line number is 1.

Thanks
Minh

0
Reply minhcao123 (2) 11/10/2006 6:36:17 PM

user101 wrote:
> Hi,  I have 2 questions
> 
> 1/ The command below prints every matched pattern blocks w/o blank RS
> record separator.
> How can I print out only FIRST and LAST pattern blocks with blank RS.
> 
> cat infile | awk ' BEGIN { RS = "" } /REBUIL/ { print } '

   awk 'BEGIN {RS="";ORS="\n\n"} /REBUIL/' <infile

> 
> 2/ The infile is 260573bytes, but awk only can read 3000bytes .

You mean the record size or the file size?
What awk are you using?
Any difference with GNU awk (or [Solaris] /usr/bin/xpg4/awk or nawk)?

> Is there any suggestion on file size limitation ?

Try this one...

   awk '/REBUIL/,/^$/' <infile


Janis

> 
> cat infile | awk ' BEGIN { RS = "" } /REBUIL/ { print } '
> awk: Input line Controller At M/C1:
>  cannot be longer than 3,000 bytes.
>  The input line number is 169.
>  The source line number is 1.
> 
> Thanks
> Minh
> 
0
Reply Janis 11/10/2006 7:04:26 PM


Thanks !

It's file size. I am using awk with hpux 11i

1/ awk 'BEGIN {RS="";ORS="\n\n"} /REBUIL/' <infile
   This one print all matched blocks with file size problems.

2/ awk '/REBUIL/,/^$/' <infile
   This command works w/o file size issues. But how to print only 1st
and last matched pattern blocks ?




Janis Papanagnou wrote:
> user101 wrote:
> > Hi,  I have 2 questions
> >
> > 1/ The command below prints every matched pattern blocks w/o blank RS
> > record separator.
> > How can I print out only FIRST and LAST pattern blocks with blank RS.
> >
> > cat infile | awk ' BEGIN { RS = "" } /REBUIL/ { print } '
>
>    awk 'BEGIN {RS="";ORS="\n\n"} /REBUIL/' <infile
>
> >
> > 2/ The infile is 260573bytes, but awk only can read 3000bytes .
>
> You mean the record size or the file size?
> What awk are you using?
> Any difference with GNU awk (or [Solaris] /usr/bin/xpg4/awk or nawk)?
>
> > Is there any suggestion on file size limitation ?
>
> Try this one...
>
>    awk '/REBUIL/,/^$/' <infile
>
>
> Janis
>
> >
> > cat infile | awk ' BEGIN { RS = "" } /REBUIL/ { print } '
> > awk: Input line Controller At M/C1:
> >  cannot be longer than 3,000 bytes.
> >  The input line number is 169.
> >  The source line number is 1.
> > 
> > Thanks
> > Minh
> >

0
Reply user101 11/10/2006 7:28:12 PM

user101 wrote:
> Thanks !

[Please don't top-post!]

> 
> It's file size. I am using awk with hpux 11i
> 
> 1/ awk 'BEGIN {RS="";ORS="\n\n"} /REBUIL/' <infile
>    This one print all matched blocks with file size problems.
> 
> 2/ awk '/REBUIL/,/^$/' <infile
>    This command works w/o file size issues. But how to print only 1st
> and last matched pattern blocks ?

Oh sorry, despite your emphasis I've missed that point.

That might do it if there wouldn't be the file size problem with your awk...

   awk '
     BEGIN {ORS="\n\n";RS=""} /REBUIL/&&!c++; /REBUIL/{s=$0} END{print s}
   ' <infile


Janis

> 
> 
> 
> 
> Janis Papanagnou wrote:
> 
>>user101 wrote:
>>
>>>Hi,  I have 2 questions
>>>
>>>1/ The command below prints every matched pattern blocks w/o blank RS
>>>record separator.
>>>How can I print out only FIRST and LAST pattern blocks with blank RS.
>>>
>>>cat infile | awk ' BEGIN { RS = "" } /REBUIL/ { print } '
>>
>>   awk 'BEGIN {RS="";ORS="\n\n"} /REBUIL/' <infile
>>
>>
>>>2/ The infile is 260573bytes, but awk only can read 3000bytes .
>>
>>You mean the record size or the file size?
>>What awk are you using?
>>Any difference with GNU awk (or [Solaris] /usr/bin/xpg4/awk or nawk)?
>>
>>
>>>Is there any suggestion on file size limitation ?
>>
>>Try this one...
>>
>>   awk '/REBUIL/,/^$/' <infile
>>
>>
>>Janis
>>
>>
>>>cat infile | awk ' BEGIN { RS = "" } /REBUIL/ { print } '
>>>awk: Input line Controller At M/C1:
>>> cannot be longer than 3,000 bytes.
>>> The input line number is 169.
>>> The source line number is 1.
>>>
>>>Thanks
>>>Minh
>>>
> 
> 
0
Reply Janis 11/10/2006 9:17:44 PM


> 1/ The command below prints every matched pattern blocks w/o blank RS
> record separator.
> How can I print out only FIRST and LAST pattern blocks with blank RS.

Do you mean the first and last matched block, or just the first and
last block?

This will do the first and last block (without matching), and shouldn't
have the line length limit you saw:

cat infile | awk '
BEGIN {b=0}
/^$/ {b=1;last=""}
b == 0 {first = first "\n" $0}
{last = last "\n" $0}
END {print "FIRST"
print first
print "LAST"
print last}'

I don't have time to do the matching form of this right now.  :(

- Sam

> cat infile | awk ' BEGIN { RS = "" } /REBUIL/ { print } '
>
> 2/ The infile is 260573bytes, but awk only can read 3000bytes .
> Is there any suggestion on file size limitation ?
>
> cat infile | awk ' BEGIN { RS = "" } /REBUIL/ { print } '
> awk: Input line Controller At M/C1:
>  cannot be longer than 3,000 bytes.
>  The input line number is 169.
>  The source line number is 1.
> 
> Thanks
> Minh

0
Reply strenholme 11/10/2006 11:55:28 PM

user101 wrote:
> Hi,  I have 2 questions
> 
> 1/ The command below prints every matched pattern blocks w/o blank RS
> record separator.
> How can I print out only FIRST and LAST pattern blocks with blank RS.
> 
> cat infile | awk ' BEGIN { RS = "" } /REBUIL/ { print } '
> 
> 2/ The infile is 260573bytes, but awk only can read 3000bytes .
> Is there any suggestion on file size limitation ?
> 
> cat infile | awk ' BEGIN { RS = "" } /REBUIL/ { print } '
> awk: Input line Controller At M/C1:
>  cannot be longer than 3,000 bytes.
>  The input line number is 169.
>  The source line number is 1.

Print first block:

     awk '/REBUIL/{f=1}f;f&&/^$/{exit}' file

Print last block (since you're on UNIX and your awk seems to be 
struggling with the record size) if you have "tac":

     tac file | awk '/^$/{f=1}f;f&&/REBUIL/{exit}' | tac

Regards,

	Ed.
0
Reply Ed 11/11/2006 2:12:46 PM

Using gawk is resolved file size problems.

Yes, I like to print 1st and last matched blocks .

Thanks !


Ed Morton wrote:
> user101 wrote:
> > Hi,  I have 2 questions
> >
> > 1/ The command below prints every matched pattern blocks w/o blank RS
> > record separator.
> > How can I print out only FIRST and LAST pattern blocks with blank RS.
> >
> > cat infile | awk ' BEGIN { RS = "" } /REBUIL/ { print } '
> >
> > 2/ The infile is 260573bytes, but awk only can read 3000bytes .
> > Is there any suggestion on file size limitation ?
> >
> > cat infile | awk ' BEGIN { RS = "" } /REBUIL/ { print } '
> > awk: Input line Controller At M/C1:
> >  cannot be longer than 3,000 bytes.
> >  The input line number is 169.
> >  The source line number is 1.
>
> Print first block:
>
>      awk '/REBUIL/{f=1}f;f&&/^$/{exit}' file
>
> Print last block (since you're on UNIX and your awk seems to be
> struggling with the record size) if you have "tac":
>
>      tac file | awk '/^$/{f=1}f;f&&/REBUIL/{exit}' | tac
> 
> Regards,
> 
> 	Ed.

0
Reply user101 11/13/2006 6:22:13 PM

Using gawk is resolved file size problems.

Yes, I like to print 1st and last matched blocks .

Thanks !


Ed Morton wrote:
> user101 wrote:
> > Hi,  I have 2 questions
> >
> > 1/ The command below prints every matched pattern blocks w/o blank RS
> > record separator.
> > How can I print out only FIRST and LAST pattern blocks with blank RS.
> >
> > cat infile | awk ' BEGIN { RS = "" } /REBUIL/ { print } '
> >
> > 2/ The infile is 260573bytes, but awk only can read 3000bytes .
> > Is there any suggestion on file size limitation ?
> >
> > cat infile | awk ' BEGIN { RS = "" } /REBUIL/ { print } '
> > awk: Input line Controller At M/C1:
> >  cannot be longer than 3,000 bytes.
> >  The input line number is 169.
> >  The source line number is 1.
>
> Print first block:
>
>      awk '/REBUIL/{f=1}f;f&&/^$/{exit}' file
>
> Print last block (since you're on UNIX and your awk seems to be
> struggling with the record size) if you have "tac":
>
>      tac file | awk '/^$/{f=1}f;f&&/REBUIL/{exit}' | tac
> 
> Regards,
> 
> 	Ed.

0
Reply user101 11/13/2006 6:28:08 PM

7 Replies
453 Views

(page loaded in 0.429 seconds)

Similiar Articles:













7/23/2012 2:46:03 AM


Reply: