PHP
Processing Large Datasets Efficiently with Eloquent `chunkById`
Handle memory-intensive tasks by processing thousands of Eloquent records in smaller chunks using `chunkById`, ensuring your application remains performant.
<?php
use App\Models\Product;
// Imagine a long-running background job or command
echo "Starting product processing...
";
Product::chunkById(500, function ($products) {
foreach ($products as $product) {
// Perform some heavy processing on each product
// e.g., update its status, calculate derived values, send to an external API
$product->status = 'processed';
$product->save();
echo "Processed Product ID: " . $product->id . "
";
}
// Optional: Log completion of a chunk or sleep for a bit
echo "Finished processing a chunk of 500 products.
";
// sleep(1); // To prevent overwhelming external services or the database
}, $column = 'id', $alias = null);
echo "Product processing completed.
";
How it works: When dealing with hundreds of thousands or millions of records, fetching them all into memory at once can lead to memory exhaustion. The `chunkById` method provides an elegant solution by retrieving records in smaller batches based on their primary key. It passes each chunk to a given callback, allowing you to process records sequentially without holding the entire dataset in memory, which is crucial for performance-critical tasks like data migrations or batch updates.