PHP
Process Large Datasets with Eloquent Chunking
Efficiently iterate and process thousands or millions of records without consuming excessive memory using Laravel Eloquent's `chunk()` and `chunkById()` methods.
<?php
use App\Models\Order;
// Example 1: Basic chunking
// Processes records in chunks of 100. Best for unsorted data or when order doesn't matter.
Order::chunk(100, function ($orders) {
foreach ($orders as $order) {
// Process each order
echo "Processing Order ID: " . $order->id . "
";
$order->status = 'processed';
$order->save();
}
});
echo "
Finished basic chunking.
";
// Example 2: Chunking by ID (recommended for large datasets, especially when modifying records)
// Ensures consistent pagination even if records are added/deleted during processing.
Order::chunkById(50, function ($orders) {
foreach ($orders as $order) {
// Process each order
echo "Processing Order ID (by ID): " . $order->id . "
";
$order->is_archived = true;
$order->save();
}
}, 'id', 'id_column_name_if_not_default'); // 'id' is default column, can specify custom
echo "
Finished chunking by ID.";
How it works: When working with a large number of database records, fetching all of them at once can exhaust your application's memory. Eloquent's `chunk()` method retrieves a small 'chunk' of records at a time, passes them to a callback, and then fetches the next chunk until all records are processed. `chunkById()` is similar but ensures that new records or deleted records during the process won't cause issues with pagination, as it explicitly fetches records with IDs greater than the last processed ID. This makes it ideal for long-running tasks like data migrations or report generation, providing significant memory efficiency.