Jul 7, 2012

Quick guide to using awk

awk allows one to look at data in a row-column format, and perform pattern matching/extraction. This works best when the structure of the data is known in advance.

Syntax:
awk '[pattern] {[action to be done on matching lines]}'
A sample pattern could be like $1="abc" ($1 => first column, $n => nth column, $0 => entire line)
Eg: Extract specific columns from ps (A better way to do this would be using ps's output formats switch. We're using awk here just as an illustration of its capabilities)
$ ps -aef
root      3558     1  0 20:14 ?        00:00:00 nginx: master process nginx
www-data  3559  3558  0 20:14 ?        00:00:00 nginx: worker process
www-data  3560  3558  0 20:14 ?        00:00:00 nginx: worker process
www-data  3561  3558  0 20:14 ?        00:00:00 nginx: worker process
www-data  3562  3558  0 20:14 ?        00:00:00 nginx: worker process

$ ps -aef | awk '$8=/.*nginx.*/ {print $2, $9} '
3558 master
3559 worker
3560 worker
3561 worker
3562 worker

Another example: Here we try to print the user names from the passwd file. The password file has ":" as its Field Separator. We convey this to awk by adding FS=":"
$ cat /etc/passwd
root:x:0:0:root:/root:/bin/bash
daemon:x:1:1:daemon:/usr/sbin:/bin/sh
bin:x:2:2:bin:/bin:/bin/sh
sys:x:3:3:sys:/dev:/bin/sh

$ cat /etc/passwd | awk ' BEGIN{FS=":"}  {print $1}'
root
daemon
bin
sys
Notice the usage of the BEGIN keyword in the above example. Even before awk sees FS=":", it reads the first line in the passwd file. Without the BEGIN keyword, the first line (root:x:0:0...) would not have been subjected to the FS=":" condition.

As a result, the first line would have been split using the default Field Separator(i.e. space) and this would yield incorrect results. By enclosing FS=":" within a BEGIN keyword, we are telling awk that this is to be processed right at the start.