JAVASCRIPT

Iterating Through All Pages of a Paginated API Response

Learn how to programmatically fetch all available data from a paginated REST API endpoint by iteratively requesting subsequent pages until no more results are available.

async function fetchAllPaginatedData(baseUrl, initialParams = {}) {
  let allData = [];
  let currentPage = 1;
  let hasMore = true;

  while (hasMore) {
    try {
      const url = new URL(baseUrl);
      for (const key in initialParams) {
        url.searchParams.append(key, initialParams[key]);
      }
      url.searchParams.append('page', currentPage);
      url.searchParams.append('limit', 100); // Assuming a limit parameter

      const response = await fetch(url.toString());
      if (!response.ok) {
        throw new Error(`HTTP error! Status: ${response.status}`);
      }
      const pageData = await response.json();

      // Assuming the API returns an array for results and an indicator for more pages
      // Adjust logic based on actual API response structure (e.g., pageData.results, pageData.next_page_url)
      if (Array.isArray(pageData.data) && pageData.data.length > 0) { // Example: pageData.data contains the items
        allData = allData.concat(pageData.data);
        currentPage++;
        // Assume 'last_page' or 'total_pages' is available for comparison, or 'next_page_url'
        // For simplicity, let's assume if data array is empty, it's the last page.
        if (currentPage > pageData.last_page) { // Example: API returns total pages
            hasMore = false;
        }
      } else {
        hasMore = false; // No more data received
      }
    } catch (error) {
      console.error('Error fetching paginated data:', error);
      hasMore = false; // Stop on error
    }
  }
  return allData;
}

// Example Usage:
// fetchAllPaginatedData('https://api.example.com/items', { category: 'electronics' })
//   .then(data => console.log('Fetched all items:', data))
//   .catch(err => console.error('Failed to fetch all items:', err));
How it works: This JavaScript snippet demonstrates how to fetch all records from a paginated API endpoint. It uses an asynchronous `while` loop, incrementing the `page` parameter with each request. It continues fetching until the API response indicates no more data (e.g., an empty array of results or a specific `last_page` indicator is reached). This pattern is useful for retrieving full datasets that are exposed through paginated APIs, commonly used in client-side data aggregation.

Need help integrating this into your project?

Our team of expert developers can help you build your custom application from scratch.

Hire DigitalCodeLabs