PYTHON
Robust API Client with Exponential Backoff for Rate Limiting
Implement a resilient API client in Python that automatically retries requests with exponential backoff to gracefully handle rate limiting and temporary network issues.
import requests
import time
def fetch_with_retry(url, headers=None, max_retries=5, initial_delay=1):
for i in range(max_retries):
try:
response = requests.get(url, headers=headers, timeout=10)
response.raise_for_status() # Raise HTTPError for bad responses (4xx or 5xx)
return response.json()
except requests.exceptions.HTTPError as e:
if response.status_code == 429 or 500 <= response.status_code < 600:
delay = initial_delay * (2 ** i)
print(f"Rate limited or server error ({response.status_code}). Retrying in {delay} seconds...")
time.sleep(delay)
else:
print(f"HTTP error: {e}")
raise
except requests.exceptions.RequestException as e:
delay = initial_delay * (2 ** i)
print(f"Network error: {e}. Retrying in {delay} seconds...")
time.sleep(delay)
raise Exception(f"Failed to fetch {url} after {max_retries} attempts")
# Example usage:
# try:
# data = fetch_with_retry("https://api.example.com/data")
# print("Fetched data:", data)
# except Exception as e:
# print("Error during API call:", e)
How it works: This Python snippet demonstrates a robust way to interact with external APIs, incorporating an exponential backoff strategy for retries. It catches `HTTPError` for status codes like 429 (Too Many Requests) or 5xx (Server Error) and `RequestException` for network issues. If such an error occurs, it waits for an exponentially increasing duration before retrying the request, up to a `max_retries` limit. This approach prevents overwhelming the API and improves the reliability of your integration by gracefully handling transient issues.