JAVASCRIPT
Iterating Through All Pages of a Paginated API Response
Learn how to programmatically fetch all available data from a paginated REST API endpoint by iteratively requesting subsequent pages until no more results are available.
async function fetchAllPaginatedData(baseUrl, initialParams = {}) {
let allData = [];
let currentPage = 1;
let hasMore = true;
while (hasMore) {
try {
const url = new URL(baseUrl);
for (const key in initialParams) {
url.searchParams.append(key, initialParams[key]);
}
url.searchParams.append('page', currentPage);
url.searchParams.append('limit', 100); // Assuming a limit parameter
const response = await fetch(url.toString());
if (!response.ok) {
throw new Error(`HTTP error! Status: ${response.status}`);
}
const pageData = await response.json();
// Assuming the API returns an array for results and an indicator for more pages
// Adjust logic based on actual API response structure (e.g., pageData.results, pageData.next_page_url)
if (Array.isArray(pageData.data) && pageData.data.length > 0) { // Example: pageData.data contains the items
allData = allData.concat(pageData.data);
currentPage++;
// Assume 'last_page' or 'total_pages' is available for comparison, or 'next_page_url'
// For simplicity, let's assume if data array is empty, it's the last page.
if (currentPage > pageData.last_page) { // Example: API returns total pages
hasMore = false;
}
} else {
hasMore = false; // No more data received
}
} catch (error) {
console.error('Error fetching paginated data:', error);
hasMore = false; // Stop on error
}
}
return allData;
}
// Example Usage:
// fetchAllPaginatedData('https://api.example.com/items', { category: 'electronics' })
// .then(data => console.log('Fetched all items:', data))
// .catch(err => console.error('Failed to fetch all items:', err));
How it works: This JavaScript snippet demonstrates how to fetch all records from a paginated API endpoint. It uses an asynchronous `while` loop, incrementing the `page` parameter with each request. It continues fetching until the API response indicates no more data (e.g., an empty array of results or a specific `last_page` indicator is reached). This pattern is useful for retrieving full datasets that are exposed through paginated APIs, commonly used in client-side data aggregation.