PHP
Processing Large Datasets with Eloquent `chunkById`
Process and iterate over millions of records in Laravel Eloquent without memory exhaustion using the `chunkById` method for efficient data handling.
use App\Models\Order;
// Process all orders in chunks of 1000, ordered by ID.
Order::chunkById(1000, function (\Illuminate\Support\Collection $orders) {
foreach ($orders as $order) {
// Perform some heavy operation on each order
$order->processPayment();
$order->update(['status' => 'processed']);
}
// Log or perform actions after each chunk is processed
echo "Processed a chunk of orders.
";
});
// Process orders older than a specific date.
Order::where('created_at', '<', now()->subMonths(6))
->chunkById(500, function (\Illuminate\Support\Collection $orders) {
foreach ($orders as $order) {
// Archive or delete old orders
$order->archive();
}
echo "Archived a chunk of old orders.
";
}, $column = 'id', $alias = null); // 'id' is default column, optional to specify
// Use 'cursor' for even lower memory footprint if you don't need to update records within the loop
// (The cursor() method only fetches a single Eloquent model at a time).
Order::where('status', 'pending')
->cursor() // Returns a Generator
->each(function (Order $order) {
// Process pending order without loading all at once
$order->notifyCustomer();
});
How it works: When dealing with extremely large datasets, fetching all records into memory at once can lead to memory exhaustion. Eloquent's `chunkById` method elegantly solves this by retrieving a subset ('chunk') of records, passing them to a callback, and then fetching the next chunk. This process repeats until all records are processed, using much less memory. The `cursor` method offers an even lower memory footprint by only executing a single query and hydrating one Eloquent model at a time, making it ideal for read-only operations on massive tables.