JAVASCRIPT

Fetching All Pages from a Paginated REST API

Learn how to programmatically fetch all data from a paginated REST API by chaining requests, ensuring you retrieve comprehensive datasets efficiently.

async function fetchAllPaginatedData(url, initialPage = 1, allItems = []) {
  try {
    const response = await fetch(`${url}?page=${initialPage}`);
    if (!response.ok) {
      throw new Error(`HTTP error! status: ${response.status}`);
    }
    const data = await response.json();

    // Assuming the API returns data in 'items' and has 'totalPages' or a 'next' link
    const items = data.items || data.results; // Adjust based on your API response structure
    const totalPages = data.totalPages || data.meta?.pagination?.pages; // Or similar
    const nextPageLink = data.next_page_link; // If using link-based pagination

    if (items && items.length > 0) {
      allItems.push(...items);
    }

    // Determine if there's a next page based on totalPages or a next link
    if (nextPageLink) {
      // For link-based pagination
      const nextUrl = new URL(nextPageLink);
      const nextPage = nextUrl.searchParams.get('page'); // Extract page number
      return fetchAllPaginatedData(url, parseInt(nextPage), allItems);
    } else if (totalPages && initialPage < totalPages) {
      // For page number-based pagination
      return fetchAllPaginatedData(url, initialPage + 1, allItems);
    } else {
      return allItems; // All pages fetched
    }
  } catch (error) {
    console.error("Error fetching paginated data:", error);
    throw error; // Re-throw to allow caller to handle
  }
}

// Example usage:
// (async () => {
//   try {
//     const allPosts = await fetchAllPaginatedData('https://api.example.com/posts');
//     console.log('All fetched posts:', allPosts);
//   } catch (error) {
//     console.error('Failed to get all posts.');
//   }
// })();
How it works: This JavaScript snippet provides a recursive `fetchAllPaginatedData` function to retrieve all available pages from a paginated REST API. It sequentially fetches each page, accumulates the items, and continues fetching until no more pages are indicated by the API response (either via a `totalPages` count or a `next_page_link`). This pattern ensures complete data retrieval from APIs that limit results per request.

Need help integrating this into your project?

Our team of expert developers can help you build your custom application from scratch.

Hire DigitalCodeLabs