BASH
Extract Unique IP Addresses from Web Server Access Logs
Create a Bash script to efficiently parse web server access logs (Apache/Nginx) and extract all unique IP addresses, useful for security analysis or traffic monitoring.
#!/bin/bash
# Configuration
LOG_FILE="/var/log/nginx/access.log" # Path to your access log file
OUTPUT_FILE="unique_ips_$(date +'%Y%m%d_%H%M%S').txt" # Output file for unique IPs
# --- Functions ---
log_message() {
echo "[$(date +'%Y-%m-%d %H:%M:%S')] $1"
}
error_exit() {
log_message "ERROR: $1"
exit 1
}
# --- Main Script ---
log_message "Starting unique IP extraction from $LOG_FILE..."
if [ ! -f "$LOG_FILE" ]; then
error_exit "Log file $LOG_FILE not found."
fi
log_message "Processing log file..."
# Regex to capture IP addresses at the beginning of the line (common for Nginx/Apache combined format)
# Using awk for efficient parsing and then sort -u for uniqueness
awk '{print $1}' "$LOG_FILE" | grep -E '^([0-9]{1,3}\.){3}[0-9]{1,3}$' | sort -u > "$OUTPUT_FILE"
if [ $? -ne 0 ]; then
error_exit "Failed to extract unique IPs. Check log file format or permissions."
else
UNIQUE_COUNT=$(wc -l < "$OUTPUT_FILE")
log_message "Unique IP addresses extracted successfully to $OUTPUT_FILE."
log_message "Total unique IPs found: $UNIQUE_COUNT"
fi
log_message "Finished unique IP extraction."
How it works: This script parses a web server access log file to find and list all unique IP addresses that accessed the server. It uses `awk` to quickly extract the first field (usually the IP), `grep` to validate the IP format, and `sort -u` to ensure only unique entries are saved to an output file. This is useful for security analysis, identifying bots, or traffic pattern analysis.