BASH

Search and Filter File Content with grep and awk

Discover how to efficiently search for patterns within file content using `grep` and perform more advanced text processing, filtering, and data extraction with `awk` in Bash.

#!/bin/bash

# Create a dummy log file for demonstration
echo "INFO: Application started at 2023-10-26 10:00:00" > app.log
echo "DEBUG: Configuration loaded from /etc/app/config.conf" >> app.log
echo "WARN: Disk usage is 85% on /var/log" >> app.log
echo "ERROR: Database connection failed for user 'admin' at 2023-10-26 10:01:15" >> app.log
echo "INFO: User 'john.doe' logged in from 192.168.1.100" >> app.log
echo "DEBUG: Processing request ID: 12345" >> app.log
echo "WARN: High memory usage detected, 92%" >> app.log
echo "ERROR: File not found: /opt/app/data.txt" >> app.log
echo "INFO: Application stopped." >> app.log

echo "--- All log entries ---"
cat app.log
echo ""

echo "--- Lines containing 'ERROR' with grep ---"
grep "ERROR" app.log
echo ""

echo "--- Lines containing 'WARN' or 'ERROR' (case-insensitive) ---"
grep -iE "WARN|ERROR" app.log
echo ""

echo "--- Extracting 'INFO' messages and their timestamps with awk ---"
# Select lines starting with 'INFO', then print the 2nd and 6th field (timestamp)
awk '/^INFO:/ { print $2, $6, $7 }' app.log
echo ""

echo "--- Filtering lines where disk usage is above 90% using awk ---"
# Searches for lines containing "usage is" and then checks if the 4th field (percentage) is > 90
awk '/usage is/ {
  gsub(/%/, "", $4); # Remove '%' from the percentage string
  if ($4 > 90) {
    print "CRITICAL: " $0; # Prepend "CRITICAL: " to the original line
  }
}' app.log
echo ""

# Clean up the dummy file
rm app.log
How it works: This snippet showcases the power of `grep` and `awk` for text processing. `grep` is used for basic pattern matching, demonstrating how to find specific strings, or multiple patterns with case-insensitivity. `awk` is used for more complex tasks: extracting specific fields (columns) from lines that match a pattern, and performing conditional logic (like checking a numerical value after string manipulation with `gsub`) to filter and format output.

Need help integrating this into your project?

Our team of expert developers can help you build your custom application from scratch.

Hire DigitalCodeLabs