awk

Updated: April 28, 2024

Used for pattern scanning. Awk has it’s own processing language. Good for separating out fields in text documents. Can transform data files and produce formatted reports.

Syntax
Arguments
Variables
Examples

SYNTAX

awk options 'selection _criteria {action }' input-file > output-file

ARGUMENTS

-f <file>		# reads program source from file instead of the first command line argument
-F fs			# use fs for the input field separator

VARIABLES

NR				# Number of input Records (records are lines, unless RS is used)
NF				# Number of Fields (within current input record)
FS				# Field Separator (character used to divide fields)
RS				# Record Separator (change from newline to something else: ; comes to mind)
OFS				# Output Field Separator (default is empty space)
ORS				# Output Record Separator (default is newline)

EXAMPLES

By Default awk prints every line of data from the speicified file. (a lot like cat)
If we want to print lines which match a certain pattern: any line with sales inside the file accounting.

awk '/sales/ {print}' accounting.txt

We can also just used awk’s built in fields. Each word on a line is given a $n variable. So the first four words in a line would be $1, $2, $3, $4 respectively while $0 represents the whole line!

awk '{print $2,$5}' accounting.txt		# print out the 2nd and 5th field of each record in the file

Since NR counts the number of record it could be used to number the lines.

awk '{print NR,$0}' accounting.txt		# makes output have line numbers

Since NF keeps track of the number of field in a record it can be used to print last field.

awk '{print $1,$NF}' accounting.txt		# prints the first field and the last of each record in the file

We can also print certain lines by number using NR

awk 'NR==18, NR==56 {print NR,$0}' accounting.txt	# prints lines 18 to 56

Print out second item along with line number separated with a -

awk '{print NR "- " $2 }' accounting.txt

Print any non empty line

awk 'NF > 0' accounting.txt

Count the lines in a file

awk 'END { print NR ' accounting.txt

Find the longest line in a file

awk '{ if (length($0) > max) max = length($0) } END { print max }' accounting.txt

Print lines with more than 8 characters

awk 'length($0) > 8' accounting.txt

Using awk within cli

# prints 5th field of ls output (file size).
ls -l | awk '{ print $5 }'

awk