PYTHON
Gracefully Handling API Rate Limits with Retry-After Header in Python
Implement a robust API client in Python that automatically retries requests based on the 'Retry-After' header to respect API rate limits effectively.
import requests
import time
def fetch_with_rate_limit(url, max_retries=3):
for attempt in range(max_retries + 1):
response = requests.get(url)
if response.status_code == 429: # Too Many Requests
retry_after = response.headers.get('Retry-After')
if retry_after:
wait_time = int(retry_after)
print(f"Rate limit hit. Retrying after {wait_time} seconds...")
time.sleep(wait_time)
else:
# If Retry-After header is missing, use a default backoff
default_wait_time = 2 ** attempt # Simple exponential backoff fallback
print(f"Rate limit hit, no Retry-After. Retrying after {default_wait_time} seconds (attempt {attempt+1})...")
time.sleep(default_wait_time)
elif 200 <= response.status_code < 300:
print(f"Successfully fetched data on attempt {attempt+1}.")
return response.json()
else:
print(f"Error: {response.status_code} - {response.text}")
break # For other errors, don't retry
print("Max retries exceeded.")
return None
# Example usage (replace with a real API endpoint):
# response_data = fetch_with_rate_limit('https://api.example.com/data')
# if response_data:
# print(response_data)
How it works: This Python snippet demonstrates how to gracefully handle API rate limiting by respecting the `Retry-After` HTTP header. When an API returns a 429 (Too Many Requests) status code, the code checks for the `Retry-After` header to determine how long to wait before retrying the request. If the header is present, it pauses execution for the specified duration using `time.sleep()`. If the header is missing, it falls back to a simple exponential backoff strategy. This ensures your application doesn't overwhelm the API and recovers automatically from temporary rate limit errors.