BASH
Download File if Not Exists or Outdated
Automate file downloads with this intelligent bash script that checks if a file exists locally or if its remote version is newer, preventing unnecessary data transfers.
#!/bin/bash
# Function to display usage
usage() {
echo "Usage: $0 <url> <destination_file>"
echo " <url>: The URL of the file to download."
echo " <destination_file>: The local path where the file should be saved."
exit 1
}
# Check for required arguments
if [ -z "$1" ] || [ -z "$2" ]; then
usage
fi
URL="$1"
DEST_FILE="$2"
DEST_DIR=$(dirname "$DEST_FILE")
# Create destination directory if it doesn't exist
mkdir -p "$DEST_DIR"
if [ -f "$DEST_FILE" ]; then
echo "Local file '$DEST_FILE' exists. Checking if remote is newer..."
# Use curl's -z (if-modified-since) to only download if remote is newer
curl -sf -z "$DEST_FILE" -o "$DEST_FILE.tmp" "$URL"
CURL_STATUS=$?
if [ $CURL_STATUS -eq 0 ]; then
# curl -z returns 0 even if no change if it doesn't actually download, but it also downloads to .tmp
# Check if the .tmp file was actually created and has content
if [ -s "$DEST_FILE.tmp" ]; then
mv "$DEST_FILE.tmp" "$DEST_FILE"
echo "File updated successfully."
else
# No new content downloaded (HTTP 304 or similar, or empty response)
rm -f "$DEST_FILE.tmp" # Clean up empty temp file
echo "Local file is already up to date or no newer version available."
fi
elif [ $CURL_STATUS -eq 33 ]; then # HTTP 304 Not Modified (curl specific error code for -z)
echo "Local file is already up to date."
else
echo "ERROR: Failed to check/update file (curl exit code: $CURL_STATUS)."
rm -f "$DEST_FILE.tmp" # Clean up potential partial temp file
exit 1
fi
else
echo "Local file '$DEST_FILE' does not exist. Downloading..."
# Directly download the file
curl -sf -o "$DEST_FILE" "$URL"
CURL_STATUS=$?
if [ $CURL_STATUS -eq 0 ]; then
echo "File downloaded successfully."
else
echo "ERROR: Failed to download file (curl exit code: $CURL_STATUS)."
exit 1
fi
fi
How it works: This script intelligently manages file downloads to prevent redundant transfers. It first checks if the target file already exists locally. If it does, it uses `curl`'s conditional download (`-z` option) to only fetch the file if the remote version is newer than the local one, based on modification dates. If the file doesn't exist locally, it performs a direct download. This is highly useful for deployment scripts, cache management, or regularly updating external resources, saving bandwidth and time.