JAVASCRIPT
Fetching All Pages from a Paginated REST API
Learn how to programmatically fetch all data from a paginated REST API by chaining requests, ensuring you retrieve comprehensive datasets efficiently.
async function fetchAllPaginatedData(url, initialPage = 1, allItems = []) {
try {
const response = await fetch(`${url}?page=${initialPage}`);
if (!response.ok) {
throw new Error(`HTTP error! status: ${response.status}`);
}
const data = await response.json();
// Assuming the API returns data in 'items' and has 'totalPages' or a 'next' link
const items = data.items || data.results; // Adjust based on your API response structure
const totalPages = data.totalPages || data.meta?.pagination?.pages; // Or similar
const nextPageLink = data.next_page_link; // If using link-based pagination
if (items && items.length > 0) {
allItems.push(...items);
}
// Determine if there's a next page based on totalPages or a next link
if (nextPageLink) {
// For link-based pagination
const nextUrl = new URL(nextPageLink);
const nextPage = nextUrl.searchParams.get('page'); // Extract page number
return fetchAllPaginatedData(url, parseInt(nextPage), allItems);
} else if (totalPages && initialPage < totalPages) {
// For page number-based pagination
return fetchAllPaginatedData(url, initialPage + 1, allItems);
} else {
return allItems; // All pages fetched
}
} catch (error) {
console.error("Error fetching paginated data:", error);
throw error; // Re-throw to allow caller to handle
}
}
// Example usage:
// (async () => {
// try {
// const allPosts = await fetchAllPaginatedData('https://api.example.com/posts');
// console.log('All fetched posts:', allPosts);
// } catch (error) {
// console.error('Failed to get all posts.');
// }
// })();
How it works: This JavaScript snippet provides a recursive `fetchAllPaginatedData` function to retrieve all available pages from a paginated REST API. It sequentially fetches each page, accumulates the items, and continues fetching until no more pages are indicated by the API response (either via a `totalPages` count or a `next_page_link`). This pattern ensures complete data retrieval from APIs that limit results per request.