BASH

Download File if Not Exists or Outdated

Automate file downloads with this intelligent bash script that checks if a file exists locally or if its remote version is newer, preventing unnecessary data transfers.

#!/bin/bash

# Function to display usage
usage() {
  echo "Usage: $0 <url> <destination_file>"
  echo "  <url>: The URL of the file to download."
  echo "  <destination_file>: The local path where the file should be saved."
  exit 1
}

# Check for required arguments
if [ -z "$1" ] || [ -z "$2" ]; then
  usage
fi

URL="$1"
DEST_FILE="$2"
DEST_DIR=$(dirname "$DEST_FILE")

# Create destination directory if it doesn't exist
mkdir -p "$DEST_DIR"

if [ -f "$DEST_FILE" ]; then
  echo "Local file '$DEST_FILE' exists. Checking if remote is newer..."
  # Use curl's -z (if-modified-since) to only download if remote is newer
  curl -sf -z "$DEST_FILE" -o "$DEST_FILE.tmp" "$URL"
  CURL_STATUS=$?

  if [ $CURL_STATUS -eq 0 ]; then
    # curl -z returns 0 even if no change if it doesn't actually download, but it also downloads to .tmp
    # Check if the .tmp file was actually created and has content
    if [ -s "$DEST_FILE.tmp" ]; then
      mv "$DEST_FILE.tmp" "$DEST_FILE"
      echo "File updated successfully."
    else
      # No new content downloaded (HTTP 304 or similar, or empty response)
      rm -f "$DEST_FILE.tmp" # Clean up empty temp file
      echo "Local file is already up to date or no newer version available."
    fi
  elif [ $CURL_STATUS -eq 33 ]; then # HTTP 304 Not Modified (curl specific error code for -z)
    echo "Local file is already up to date."
  else
    echo "ERROR: Failed to check/update file (curl exit code: $CURL_STATUS)."
    rm -f "$DEST_FILE.tmp" # Clean up potential partial temp file
    exit 1
  fi
else
  echo "Local file '$DEST_FILE' does not exist. Downloading..."
  # Directly download the file
  curl -sf -o "$DEST_FILE" "$URL"
  CURL_STATUS=$?

  if [ $CURL_STATUS -eq 0 ]; then
    echo "File downloaded successfully."
  else
    echo "ERROR: Failed to download file (curl exit code: $CURL_STATUS)."
    exit 1
  fi
fi
How it works: This script intelligently manages file downloads to prevent redundant transfers. It first checks if the target file already exists locally. If it does, it uses `curl`'s conditional download (`-z` option) to only fetch the file if the remote version is newer than the local one, based on modification dates. If the file doesn't exist locally, it performs a direct download. This is highly useful for deployment scripts, cache management, or regularly updating external resources, saving bandwidth and time.

Need help integrating this into your project?

Our team of expert developers can help you build your custom application from scratch.

Hire DigitalCodeLabs