BASH
Extract and Count Specific Patterns in Log Files
Analyze web server or application logs by extracting and counting occurrences of specific patterns, like error messages or IP addresses, using simple Bash tools.
#!/bin/bash
LOG_FILE="$1"
SEARCH_PATTERN="$2"
# Check for required arguments
if [ -z "$LOG_FILE" ] || [ -z "$SEARCH_PATTERN" ]; then
echo "Usage: $0 <log_file> <search_pattern>"
echo "Example: $0 /var/log/nginx/access.log \"GET /api/v1/user\""
exit 1
fi
# Validate if the log file exists
if [ ! -f "$LOG_FILE" ]; then
echo "Error: Log file '$LOG_FILE' not found." >&2
exit 1
fi
echo "Searching for '$SEARCH_PATTERN' in '$LOG_FILE'...
"
# Count total occurrences of the pattern
COUNT=$(grep -c "$SEARCH_PATTERN" "$LOG_FILE")
echo "Total occurrences of pattern: $COUNT"
# Show unique lines containing the pattern
echo "
Unique lines containing the pattern (first 10):"
grep "$SEARCH_PATTERN" "$LOG_FILE" | sort -u | head -n 10
# Example: If the pattern is an IP address, count occurrences of each unique IP
# Basic regex for IPv4 address
if [[ "$SEARCH_PATTERN" =~ ^[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}\.[0-9]{1,3}$ ]]; then
echo "
Counting occurrences of each unique matching IP address (top 10):"
grep -oE "$SEARCH_PATTERN" "$LOG_FILE" | sort | uniq -c | sort -nr | head -n 10
elif [[ "$SEARCH_PATTERN" =~ ^[A-Za-z0-9._%+-]+@[A-Za-z0-9.-]+\.[A-Za-z]{2,}$ ]]; then
echo "
Counting occurrences of each unique matching email address (top 10):"
grep -oE "$SEARCH_PATTERN" "$LOG_FILE" | sort | uniq -c | sort -nr | head -n 10
fi
How it works: This powerful Bash script aids in log analysis by taking a log file and a search pattern as arguments. It first counts all occurrences of the pattern, then lists unique lines containing it. Additionally, it includes conditional logic to specifically count occurrences of individual IP addresses or email addresses if the search pattern matches a basic regex for either, providing a quick way to identify frequent actors or problematic entries in logs.