AWK

AWK is a scripting language that can be used with the awk command, which is a standard feature of most Unix-like operating systems. It is used to analyze, filter, and manipulate structured data. It scans the input line by line and performs actions based on your input command. Awk is mentioned in the same breath as Sed because they are both known for being highly versatile command line power tools.

How can you use Awk?

The basic syntax for using awk in the shell is:

 awk [options] 'pattern {action}' input_file

some of the available options in awk are:

Options	Description
-F	Change the custom field separator, by default fields are seperated by spaces.
-f	Changes the awk command to run from a specified script instead.
-v	Allows you to define variables before awk processesing.

Lengths separated by the field separator, or ‘field variables’, are automatically assigned to the variable $(int) in sequential order starting at $1, where $0 prints the whole document.

AWK has many built in variables in our examples below we use NR: to represent the total number of input records seen so far. you can even access command line arguments in AWK using ARGC and ARGV. The awk man pages are a great resource to see a detailed command and variable list.

Example use cases of AWK.

  id first_name last_name gpa major
  1 John Doe 3.4 Marketing
  2 Ann Chovery 3.0 Enviromental Sciences
  3 Steve Jobs 2.9 Kinesoliogy
  4 Frank Stein 4.0 Philosphy
  5 Lois Lane 3.9 Journalism
  6 Liz Erd 3.6 Computer Science

1. Printing an entire document.

Input Command: awk '{print}' example.txt

id first_name last_name gpa major
1 John Doe 3.4 Marketing
2 Ann Chovery 3.0 Enviromental Sciences
3 Steve Jobs 2.9 Kinesoliogy
4 Frank Stein 4.0 Philosphy
5 Lois Lane 3.9 Journalism
6 Liz Erd 3.6 Computer Science

2. Printing with a different field seperator.

lets say for example we are working with an example.csv that is nearly identical to example.txt except it uses , to seperate its fields.

Input Command: awk -F "," '{print $1" "$5}' example.csv

id major
1 Marketing
2 Enviromental Sciences
3 Kinesoliogy
4 Philosphy
5 Journalism
6 Computer Science

3. Comparing with a variable.

Input Command: awk -v gpa=3.0 '$4 > gpa {print}' example.txt

id first_name last_name gpa major
1 John Doe 3.4 Marketing
4 Frank Stein 4.0 Philosphy
5 Lois Lane 3.9 Journalism
6 Liz Erd 3.6 Computer Science

4. Printing with a script. given an example script:

Input Command: awk -f awk_test.sh 

John has a 3.4 gpa!
Ann has a 3.0 gpa!
Steve has a 2.9 gpa!
Frank has a 4.0 gpa!
Lois has a 3.9 gpa!
Liz has a 3.6 gpa!

What makes Awk unique?

AWK seems to be a useful way to extract data but why wouldn’t use less complex methods. For example a combination of the grep command and the cut command.

Input Command: grep " 3.*" example.txt |cut -d ' ' -f 1,2,4

1 John 3.4
2 Ann 3.0
5 Lois 3.9
6 Liz 3.5

This combination of commands is the equivalent of the Awk command:

Input Command: awk ' $4 > 3 {print $1" "$2" "$4}' example.txt

But besides the upsides of letting us both search and extract data, it also has a unique ability to transform data within the same command. Below is the sensor; this tells us about the state of various sensors around our computer. By piping this to the AWK processor, we can run comparisons and customize the print based on features of each line. For this example, we are seeing whether any of our cores are overheating.

Input Command: sensors | awk 'NR > 3 && NR < 24 { if (int($3) > int($6)) print $1, $2, "does exceed", $6, "it is overheating"; else print $1, $2, "doesn''t exceed", $6, "it is not overheating"}'

Core 0: doesnt exceed +80.0°C, it is not overheating
Core 4: doesnt exceed +80.0°C, it is not overheating
Core 8: does exceed +80.0°C, it is overheating
Core 12: doesnt exceed +80.0°C, it is not overheating
Core 16: doesnt exceed +80.0°C, it is not overheating
Core 20: doesnt exceed +80.0°C, it is not overheating
Core 24: doesnt exceed +80.0°C, it is not overheating
Core 28: doesnt exceed +80.0°C, it is not overheating
Core 32: does exceed +80.0°C, it is overheating
Core 33: doesnt exceed +80.0°C, it is not overheating
Core 34: doesnt exceed +80.0°C, it is not overheating
Core 35: doesnt exceed +80.0°C, it is not overheating
Core 36: doesnt exceed +80.0°C, it is not overheating
Core 37: doesnt exceed +80.0°C, it is not overheating
Core 38: does exceed +80.0°C, it is overheating
Core 39: doesnt exceed +80.0°C, it is not overheating
Core 40: doesnt exceed +80.0°C, it is not overheating
Core 41: doesnt exceed +80.0°C, it is not overheating
Core 42: does exceed +80.0°C, it is overheating
Core 43: doesnt exceed +80.0°C, it is not overheating

Additional Resources.

AWK is scripting language and helpful as a data extraction and reporting tool. To better understand on how to use the awk command itself, make sure to play around in your own enviroment and consult the awk man pages.