Day 7: Mastering the awk command

Adrian Rubico

|

Aug 16, 2024

•

08:31 PM GMT+8

The awk command is a powerful text-processing tool commonly used in Unix-like systems for pattern scanning and data extraction. Whether you are filtering, formatting, or transforming text files, awk command provides an efficient and flexible way to manipulate data. Our goal in this blog is to explore what awk command can do for your Bash scripts and how we can use it.

What can we make from awk command?

The awk command is particularly used for:

  • Text processing: Extracting and manipulating text from files or command output.
  • Data transformation: Formatting data into readable output.
  • Pattern matching: Searching for patterns within text and performing actions on matches.

Let's examine into the specific operations and examples:

Basic awk Operations

The awk has functions connected to reading input line by line, dividing each line into fields (whitespace is the default delimiter), and sending commands to those fields. 

Example: Print Specific Fields 

bash 
echo "Juan Dela Cruz 25" | awk '{print $1, $4}'

This command prints the first($1) and forth($4) fields, outputting "Juan 25".

Example: Sum a Column of Numbers

bash 
awk '{sum += $1} END {print sum}' numbers.txt

This command reads a file called numbers.txt sums all the values in the first column, and prints the total.

Useful Applications of awk

It comes to more complex text processing tasks.

Example: Extract and Sort Data

bash 
awk '/red/ {print $0}' data.txt | sort -r

This command finds lines containing "red" in data.txt, prints entire lines, and sorts the results.

Example: Format CSV Files

bash 
awk -F, '{print $1, $3}' data.csv

This command reads a CSV file and prints the first and third columns.

awk with if and else Conditions

awk supports conditional statements, making it possible to execute different actions based on the data being processed.

Example: Filter Based on Condition

bash 
awk '{if ($3 > 50) print $1, $3; else print $1, "Too low"}' data.txt

This command checks if the third field is greater than 50 and prints the first and third fields if true; otherwise, it prints "Too low".

awk with Loops

We can also use loops within awk to handle repetitive tasks.

Example: Iterate Over Fields

bash 
awk '{for (i = 1; i <= NF; i++) print $i}' example_data.txt

This command prints each field in the current line, one per line.

Advanced Usage of awk with grep and sed

Combining awk with other text-processing tools like grep and sed can lead to powerful one-liners.

Example: Extract and Replace Text

bash 
grep "pattern" fruits_data.txt | awk '{print $2}' | sed 's/old/new/'

Here is the breakdown:

  • grep "red" data.txt finds two lines: apple 10 red and cherry 30 red.
  • awk '{print $2}' extracts the second field from these lines, resulting in 10 and 30.
  • sed 's/0/00/' then replaces the 0 in 10 with 00, resulting in 100, and similarly replaces the 0 in 30 with 00, resulting in 300.

Mastering awk for Efficient Text Processing

Text and data processing on the command line is impossible without the awk command. Using sed together with grep and sed can help you significantly enhance text-processing capabilities once you master its basic operations.

Our next blog will explore how the Linux package manager and systemctl can be used to manage software and services on your system. More updates on the way!

TAGS:

#90DaysDevOpsChallenge linux

Discussion