printing the first line in many files that meets a certain condition

  • Permalink
  • submit to reddit
  • Email
  • Follow


I'd like to print the first line from each of many files in which a
particular field meets a certain condition.  For example, I'd like to
print the lines that, going from top, contain the first negative value
in column 2 (line 3 in the exmaple below):

1.5992E-02, 1.6482E-03
1.6306E-02, 7.1307E-04
1.6656E-02, -3.9010E-05
1.7031E-02, -6.4005E-04
1.7437E-02, -1.1658E-03

What if I wanted to print all the lines which contain the last
positive value in column 2?  I know I would have to evaluate and store
the previous value, but have not been able to write a working
script...

Can it be done using an awk one-liner?

--z.entropic
0
Reply z 3/19/2009 12:32:24 PM

See related articles to this posting


On Thursday 19 March 2009 13:32, z.entropic wrote:

> I'd like to print the first line from each of many files in which a
> particular field meets a certain condition.  For example, I'd like to
> print the lines that, going from top, contain the first negative value
> in column 2 (line 3 in the exmaple below):
> 
> 1.5992E-02, 1.6482E-03
> 1.6306E-02, 7.1307E-04
> 1.6656E-02, -3.9010E-05
> 1.7031E-02, -6.4005E-04
> 1.7437E-02, -1.1658E-03

awk '$2<0 {print;exit}'

> 
> What if I wanted to print all the lines which contain the last
> positive value in column 2?  I know I would have to evaluate and store
> the previous value, but have not been able to write a working
> script...

awk '$2>0 {lastpos=$0} END{print lastpos}'

0
Reply pk 3/19/2009 1:06:56 PM

z.entropic <subPlanck@excite.com> wrote:
> I'd like to print the first line from each of many files in which a
> particular field meets a certain condition.  For example, I'd like to
> print the lines that, going from top, contain the first negative value
> in column 2 (line 3 in the exmaple below):
> 
> 1.5992E-02, 1.6482E-03
> 1.6306E-02, 7.1307E-04
> 1.6656E-02, -3.9010E-05
> 1.7031E-02, -6.4005E-04
> 1.7437E-02, -1.1658E-03

FNR == 1 { p = 0 }
p != 0   { next }
$2 < 0   { print $0 ; p = 1 }

> What if I wanted to print all the lines which contain the last
> positive value in column 2?  I know I would have to evaluate and store
> the previous value, but have not been able to write a working
> script...

FNR == 1 && h != "" { print h ; h = "" }
$2 > 0              { h = $0 }
END                 { if (h != "") print h }
0
Reply dave 3/19/2009 4:56:09 PM

Dave Gibson <dave+news002@gibson-hrd.abelgratis.co.uk.invalid> wrote:
> z.entropic <subPlanck@excite.com> wrote:
[...]
>> What if I wanted to print all the lines which contain the last
>> positive value in column 2?  I know I would have to evaluate and store
>> the previous value, but have not been able to write a working
>> script...
> 
> FNR == 1 && h != "" { print h ; h = "" }
> $2 > 0              { h = $0 }

Correction: 0 is positive

  !($2 < 0)           { h = $0 }
> END                 { if (h != "") print h }
0
Reply dave 3/19/2009 5:04:54 PM

On Mar 19, 9:06=A0am, pk <p...@pk.invalid> wrote:
> On Thursday 19 March 2009 13:32, z.entropic wrote:
>
> > I'd like to print the first line from each of many files in which a
> > particular field meets a certain condition. =A0For example, I'd like to
> > print the lines that, going from top, contain the first negative value
> > in column 2 (line 3 in the exmaple below):
>
> > 1.5992E-02, 1.6482E-03
> > 1.6306E-02, 7.1307E-04
> > 1.6656E-02, -3.9010E-05
> > 1.7031E-02, -6.4005E-04
> > 1.7437E-02, -1.1658E-03
>
> awk '$2<0 {print;exit}'
>
>
>
> > What if I wanted to print all the lines which contain the last
> > positive value in column 2? =A0I know I would have to evaluate and stor=
e
> > the previous value, but have not been able to write a working
> > script...
>
> awk '$2>0 {lastpos=3D$0} END{print lastpos}'

Thaks, but these expressions print only the first occurrence...  they
don't loop over many files.

--z.e.
0
Reply z 3/19/2009 8:43:50 PM

On Thursday 19 March 2009 21:43, z.entropic wrote:

> On Mar 19, 9:06 am, pk <p...@pk.invalid> wrote:
>> On Thursday 19 March 2009 13:32, z.entropic wrote:
>>
>> > I'd like to print the first line from each of many files in which a
>> > particular field meets a certain condition.  For example, I'd like to
>> > print the lines that, going from top, contain the first negative value
>> > in column 2 (line 3 in the exmaple below):
>>
>> > 1.5992E-02, 1.6482E-03
>> > 1.6306E-02, 7.1307E-04
>> > 1.6656E-02, -3.9010E-05
>> > 1.7031E-02, -6.4005E-04
>> > 1.7437E-02, -1.1658E-03
>>
>> awk '$2<0 {print;exit}'

with GNU awk:

awk '$2<0 {print;nextfile}' path/to/files/*

otherwise you can use Dave Gibson's code.

>> > What if I wanted to print all the lines which contain the last
>> > positive value in column 2?  I know I would have to evaluate and store
>> > the previous value, but have not been able to write a working
>> > script...
>>
>> awk '$2>0 {lastpos=$0} END{print lastpos}'
> 
> Thaks, but these expressions print only the first occurrence...  they
> don't loop over many files.

awk 'FNR==1&&lastpos{print lastpos} $2>0 {lastpos=$0} END{print lastpos}' *

0
Reply pk 3/19/2009 9:00:23 PM

On Mar 19, 12:56=A0pm, dave+news...@gibson-hrd.abelgratis.co.uk.invalid
(Dave Gibson) wrote:
> z.entropic <subPla...@excite.com> wrote:
> > I'd like to print the first line from each of many files in which a
> > particular field meets a certain condition. =A0For example, I'd like to
> > print the lines that, going from top, contain the first negative value
> > in column 2 (line 3 in the exmaple below):
>
> > 1.5992E-02, 1.6482E-03
> > 1.6306E-02, 7.1307E-04
> > 1.6656E-02, -3.9010E-05
> > 1.7031E-02, -6.4005E-04
> > 1.7437E-02, -1.1658E-03
>
> FNR =3D=3D 1 { p =3D 0 }
> p !=3D 0 =A0 { next }
> $2 < 0 =A0 { print $0 ; p =3D 1 }
>
> > What if I wanted to print all the lines which contain the last
> > positive value in column 2? =A0I know I would have to evaluate and stor=
e
> > the previous value, but have not been able to write a working
> > script...
>
> FNR =3D=3D 1 && h !=3D "" { print h ; h =3D "" }
> $2 > 0 =A0 =A0 =A0 =A0 =A0 =A0 =A0{ h =3D $0 }
> END =A0 =A0 =A0 =A0 =A0 =A0 =A0 =A0 { if (h !=3D "") print h }

Thanks, that's almost it.  However, since I like one-liners, how do I
use it as a line command, not an awk script? That is, the following
works:

$ ls *.z|xargs -l awk -f fn.awk

where fn.awk is

FNR =3D=3D 1 { p =3D 0 }; p !=3D 0 { next }; NR > 10 && $6 < 0 { print
FILENAME, $0; p =3D 1}

but this does not as it prints only the first occurrence...

$ cat *.z | awk 'FNR =3D=3D 1 { p =3D 0 }; p !=3D 0 { next }; NR > 10 && $6=
 <
0 { print FILENAME, $0; p =3D 1 }'

--z.e.
0
Reply z 3/19/2009 9:01:31 PM

z.entropic <subPlanck@excite.com> wrote:
[...]
> Thanks, that's almost it.  However, since I like one-liners, how do I
> use it as a line command, not an awk script? That is, the following
> works:
> 
> $ ls *.z|xargs -l awk -f fn.awk
> 
> where fn.awk is
> 
> FNR == 1 { p = 0 }; p != 0 { next }; NR > 10 && $6 < 0 { print
> FILENAME, $0; p = 1}

Either:

awk -f fn.awk *.z

Or insert something like this:

#! /usr/bin/awk -f

as the first line of the script (change the path to match whichever
awk you're using), then chmod 755 the script and:

fn.awk *.z

> but this does not as it prints only the first occurrence...
> 
> $ cat *.z | awk 'FNR == 1 { p = 0 }; p != 0 { next }; NR > 10 && $6 <
> 0 { print FILENAME, $0; p = 1 }'

Cat writes the contents of its arguments to standard output.  To let awk
see the filenames, use:

awk '...' *.z

Incidentally, if you want to skip the first ten lines of each input
file the "NR > 10" part of your script should be "FNR > 10".
0
Reply dave 3/19/2009 9:47:02 PM
comp.lang.awk 3235 articles. 6 followers. Post

7 Replies
214 Views

Similar Articles

[PageSpeed] 30


  • Permalink
  • submit to reddit
  • Email
  • Follow


Reply:

Similar Artilces:

What is the command to print out some certain lines of a text file?
Hi, I have an impression some unix command can print some lines of a file, for example, I want to see on my console the lines between 500 and 520, but I can not remember how to do that. Any pointer? Thanks. On 2007-10-25, linq936 wrote: > I have an impression some unix command can print some lines of a > file, for example, I want to see on my console the lines between 500 > and 520, but I can not remember how to do that. sed -n '500,520p' FILENAME If it's a very large file, you will want to exit after the last line: sed -n -e '500,520p' -e '520...

Print the section from a file from sed except first or last line
HI, sed '/AAA/,/BBB/p' <file> gives me AAA some text some text BBB Now I want is the out put without AAA or BBB. How is this possible with a single sed command? I tried following but didn't work - sed '/AAA/,/BBB/{ 1d; $d; p; }' <file> On 2009-11-18, #! /shell/nerd wrote: > HI, > > sed '/AAA/,/BBB/p' <file> gives me > > AAA > some text > some text > BBB > > > Now I want is the out put without AAA or BBB. How is this possible > with a single sed command? I tried following but didn'...

Open file, get first line, delete first line close file
Hey, i'm trying to open a file, get the first line of the file, delete that line from the file, and then close the file. Using ruby 1.8.6. I've tried using The_File = IO.readlines("public/languages/chinese/practice.txt") & The_File = open('public/languages/chinese/practice.txt','r+') but i can't figure out the correct syntax for how to delete a line in a file, and then save that file. -- Posted via http://www.ruby-forum.com/. Richard Schneeman wrote: > Hey, i'm trying to open a file, get the first line of the file, delete > that line ...

print to file line by line
Is anybody know how to output results to a file line by line, since I do not have the whole matrix before printing to file? Thanks a lot. Lirong TAN wrote: > Is anybody know how to output results to a file line by line, since I do > not have the whole matrix before printing to file? > > Thanks a lot. doc fprintf % and friends comes to mind. What's the problem? -- ...

write lines of one file to new file dependent on first field of line
sample data qaaeae01: Name: ethernet qaaeae01: Node: ethernet@4,1 qaaeae01: Device Type: network qaaeae01: Physical Location: U787B.001.DNW9929-P1-C1-T2 qaaeae01: qaaeae01: Name: ethernet qaaeae01: Node: ethernet@6 qaaeae01: Device Type: network qaaeae01: Physical Location: U787B.001.DNW9929-P1-C1-T3 qaaeae01: qaaeae01: Name: ethernet qaaeae01: Node: ethernet@6,1 qaaeae01: Device Type: network qaaeae01: Physical Location: U787B.001.DNW9929-P1-C1-T4 qaaiap02: INSTALLED RESOURCE LIST WITH VPD qaaiap02: qaaiap02: The following resources are i...

First Lines and Last Lines of a File
Hi all, when i need the first N lines of a file I use awk this way: cat file | c=3 awk ' BEGIN { c=ENVIRON["c"] } NR <= c { print } ' (just an example) but can I do when I need the n first lines of a file and the last n lines of a file ? I tought of keeping n lines in an array and remove the older line after every new, pratically a FIFO structure... but it begins to be a bit difficult... anyone with a quicker/easier one? (I know I could use a combination of "tail" & "head".. but I'd like to keep processing inside a single awk "statemen...

insert a line before the first line of a file
I want to insert a line before the first line of the file, how can I do this? Thanks. abai On Mar 20, 7:42 am, "Bin Chen" <binary.c...@gmail.com> wrote: > I want to insert a line before the first line of the file, how can I > do this? > > Thanks. > > abai Could do something line sed -e '1 i Text of the first line' file > newfile or echo "Text of new line"| cat - file > newfilename Oliver Bin Chen wrote: > I want to insert a line before the first line of the file, how can I > do this? > > Thanks. > > aba...

how to print files with line number for each line?
Hi, I am wondering if there is a way to print out files with line number printed as well. Right now, I can view the line number in vim editor. I hope those line numbers are shown in the print out too. My favorite printing command is a2ps. Thank you. In news:ejaol6$ihc$1@news.nems.noaa.gov, Shawn <shaw@nospam.com> wrote: > I am wondering if there is a way to print out files with line number > printed as well. cat -n filename On 2006-11-13, Shawn wrote: > Hi, > > I am wondering if there is a way to print out files with line number > printed as well. Right now, ...

When the first line of a file tells something about the other lines
Often the first line of a file tells how to read or interpret the other lines. Depending on the result, you then have to ... - skip the first line, or - treat the first line in another special way, or - treat the first line in the same way as the other lines. I can handle this by opening the file twice, the first time for reading the first line only. I suppose there exists a more elegant solution. Below is the structure of what I do now. Please comment. f = open(file_name,"r") # eerste opening file_line = f.readline() special = True if some_condition else Fal...

Insert a line at the first line without a # in a file
Hi, I have a few thousand files that are a proprietary macro language from a database. the format of the files are there are N number of lines at the top of the file that are perpended with a # for a comment symbol I want to insert a blank line and then a line of text then another blank line after the first line from the top of the file that does not have # as the first character on the line. The data format is # line 1 some text # some more comments # yet again and may be any number text text text # another comment text The data format I want is: # line 1 some text # some more comments...

Print all lines except of first (last) n lines.
Hello, I would like to write a small awk script which prints all lines of some file with the exception of the first "n" lines and the last "m" lines. I just wrote first part of this script, which does not print first "n" lines but prints another ones. BEGINNING{i=0} i<=3 {i=i+1} i>3 {print $0} In the above example "n=3". However, the following example does not work. Do you know why? BEGINNING{n=3;i=0} i<=n {i=i+1} i>n {print $0} And the last question is what I need to do with the second part of my script (which stops printing of the la...

Add a line as the first line in a file using a script
Hello, I want to add a (known) line to a file (named myFile.txt) ***as the first line*** in a BASH script To add text after the end of a file, all I have to do in the script is: cat "my text" >> myFile.txt Is it possible to add this text as the first line of this file (in a BASH script)? Regards, IB ianbrn@gmail.com wrote: > Hello, > I want to add a (known) line to a file (named myFile.txt) ***as the > first line*** > in a BASH script > > > To add text after the end of a file, all I have to do in the > script is: > > cat "my text&q...

How to print all lines of a file starting from line number X
Hi, with which command can I print all lines after a certain line number. I know that with head I can display the first X and with tail the last Y. How do print all lines which start at a certain line until the last line? Thanks, Fritz On 2006-07-12, fritz-bayer@web.de <fritz-bayer@web.de> wrote: > How do print all lines which start at a certain line until the last > line? man tail -- Grant Edwards grante Yow! Is this ANYWHERE, at USA? visi.com fritz-b...

print PDF file from command line to postscript file
How do I print a PDF file from the command line so that: 1. It prints only page 34 and nothing else. 2. Prints page 34 to a PostScript called "34.ps" I would like to script this so that I can run it from an VT100 terminal. In article <b184cb3d.0404221743.7730ed01@posting.google.com>, luther wrote: > How do I print a PDF file from the command line so that: > > 1. It prints only page 34 and nothing else. > 2. Prints page 34 to a PostScript called "34.ps" > > I would like to script this so that I can run it from an VT100 terminal. ...

Program for retrieving certain lines in a file and writing them to another file
Hi all, I am trying to write a program that scans a large log file and only extracts the lines that pertain to an error message and writes it to a different line. Basically the log file will be of this format : Junk Junk ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~``` ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~``` Error.............................................\n Junk Junk Junk~~~~~~~~~~~~~~~~~~~~~~~~~~~~```` Error..............\n So between the junk text there will be these long Error strings that are terminated by a new line. Now I know how to open the actual file and also looked up ...

How to Remove the First Line of a File, lines containing Alphabetic characters?
Dear Peers, How are you? I have a problem. I am not sure if what I am doing is correct in C/UNIX. Could someone help me? I have a file: 1-I want to remove the first line of that file Should I use the cut, grep, head, ... command? 2-I want to remove any lines containing Alphabetic characters (a, b, c, ...) All the lines with Alpha characters contain a R. What about using: grep -v "R"; Any other suggestions? 3-Change data from a matrix type of format into vectors. 0 0 0 0 100 120 250 0 0 0 0 150 170 200 0 0 0 90 170 210 300 Into: 0 0 0 0 100 1...

How to print line 3 of a file if current position is line 7
Hi, I need a pattern to find from a file. once I find that pattern I need to print previous 3 lines of the file as to the current line containing the pattern. I am using getline(), but not able to go back to the previous lines. please help. Thanks. -- [ See http://www.gotw.ca/resources/clcm.htm for info about ] [ comp.lang.c++.moderated. First time posters: Do this! ] dalu.gelu@gmail.com writes: > I need a pattern to find from a file. once I find that pattern I > need to print previous 3 lines of the file as to the current line > containing the pattern. > &g...

urgent:...read the first line of a file to use it in a bat-file
Hi! I have got one file to download every 10 min from a ftp-server. Unfortunately the name changes every time. I think about scheduling a bat-file which generates the ftp-script I need for the correct download. At the moment I stick at that point. (I know that this point would be no problem in any unix, but I�ve got no real solution for windowsNT) I generate a file which contains the result of dir /b/o-g-d in that file I get the name of the youngerst file in the directory in the first row without any other information bevore. Now I want to pick that first row and use it gene...

Extracting lines from a text file that match a certain criteria to another text file
Hi, I am novice UNIX shell programmer and would like some help. I am trying to write a shell script that will read a text file line by line. The task is to extract lines from the text file that meet a set of criteria to another text file. An example of the text file, foo2, that I am using is shown below, 0,0,"NBSC",12,"24/02/2004 15:35:34","RD","2.2" 1,1,"ORG-UNIT-TYPE","C",1,"13/03/1997 17:24:49","PO" 1,2,"ORG-UNIT-TYPE","C",2,"13/03/1997 17:24:49","Client" 1,3,"ORG-UN...

print certain lines....
I have a text file.... --------------------------------------------------------------------------------- 82 07/05/2009 DB C 85 433792 07/05/2009 2160.00 .00 ----> I want this line TOTAL AMT Net amount CCC NO:011962 Date: 31032009 ----->I want thisline 983 07/05/2009 DB D 86 433792 07/05/2009 27840.00 .00 AMOUNT CARRIED Net amount CCC NO:011413 Date: 30032009 -------------------------------------------------------------------------------- 1776 30/04/2009 DB C 87 415223 07/05/2009 4137.00 .00 -----&g...

Finding one certain line in a file is easy but how to look for headlines and something just under this certain headline?
Hi, I'm writing a little script for installing ati-drivers on linux machines but I have a little problem. I need to change the /etc/X11/xorg.conf file for this. This file is built like this (just a cut-out): Section "InputDevice" Identifier "Generic Keyboard" Driver "kbd" Option "CoreKeyboard" Option "XkbRules" "xorg" Option "XkbModel" "pc105" Option "XkbLayout" "de" Optio...

ow can i use fgets to read and ignore the first two lines of a file and output into another file
Novice programmer needs help with using fgets to read and ignore the first two lines of a file. I've gone thru the previous posting regarding fgets, but none of them seems to help my situation. I have airdata file that i have to read, but in other teh fscanf to work properly, i need to ignore the first two lines, because scanf does not read spaces. This is what my current code looks like #include <stdio.h> #include <sting.h> #include <stdlib.h> int main() { FILE * AirFile; //the file that contains the data to be read FILE *textFile; //the out file for both...

counting how often the same word appears in a txt file...But my code only prints the last line entry in the txt file
Hi Iam just starting out with python...My code below changes the txt file into a list and add them to an empty dictionary and print how often the word occurs, but it only seems to recognise and print the last entry of the txt file. Any help would be great. tm =open('ask.txt', 'r') dict = {} for line in tm: line = line.strip() line = line.translate(None, '!"#$%&\'()*+,-./:;<=>?@[\\]^_`{|}~') line = line.lower() list = line.split(' ') for word in list: if word in dict: count = dict[word] count += 1 dict[word] = coun...

Delete first line from file
Hi How can I read the first line of a file and then delete this line, so that line 2 is line 1 on next read? regards On Tue, Mar 01, 2005 at 01:27:27PM +0100, Tor Erik S?nvisen wrote: > Hi > > How can I read the first line of a file and then delete this line, so that > line 2 is line 1 on next read? > > regards > > I think you can do something like: n=false f=file.open("") #stuff here g=[] for line in f.readlines(): if n: g.append(line) n=true #write g to file if you are on a unix box, then using the standard untils might be a better idea. ...