PHP
Efficiently Process Large Datasets with Eloquent Chunking
Learn how to process large result sets from your Laravel Eloquent models without exhausting memory using `chunk` and `chunkById` for efficient batch operations.
<?php
namespace App\Models;
use Illuminate\Database\Eloquent\Model;
class Order extends Model
{
protected $fillable = ['total_amount', 'status'];
}
// Scenario: Update status for all completed orders, potentially millions of records
// Method 1: Using 'chunk' (less efficient for very large tables as offset can be slow)
App\Models\Order::where('status', 'pending')
->chunk(1000, function ($orders) {
foreach ($orders as $order) {
$order->status = 'processed';
$order->save();
}
});
echo "Orders processed using chunk.
";
// Method 2: Using 'chunkById' (more efficient for very large tables, uses primary key for offset)
// This is generally preferred for large datasets.
App\Models\Order::where('status', 'pending')
->chunkById(1000, function ($orders) {
foreach ($orders as $order) {
$order->status = 'processed_by_id';
$order->save();
}
});
echo "Orders processed using chunkById.
";
// Note: You can also use 'cursor()' for truly streaming results, but it returns PHP stdClass objects.
// App\Models\Order::where('status', 'pending')->cursor()->each(function ($order) {
// // Process each order object individually
// });
How it works: When dealing with large datasets, fetching all records into memory at once can lead to performance issues and memory exhaustion. Eloquent's `chunk()` and `chunkById()` methods provide a way to retrieve and process records in smaller batches. `chunk()` uses `LIMIT` and `OFFSET`, which can become slow for very large offsets. `chunkById()` is generally more efficient as it uses the primary key to keep track of the position, making it suitable for processing millions of records. These methods are invaluable for background jobs, data migrations, or reporting.