PHP
Processing Large Datasets with Eloquent Chunking
Learn how to efficiently process large numbers of Eloquent records in Laravel using chunking methods to prevent memory exhaustion and improve performance.
<?php
namespace App\Console\Commands;
use Illuminate\Console\Command;
use App\Models\User;
class ProcessLargeUsers extends Command
{
protected $signature = 'app:process-large-users';
protected $description = 'Process a large number of user records efficiently.';
public function handle()
{
$this->info('Starting to process users...');
User::chunk(1000, function ($users) {
foreach ($users as $user) {
// Perform operations on each user
$user->email = strtolower($user->email);
$user->save();
// Optionally, add more complex logic or dispatch jobs
// ProcessUserJob::dispatch($user);
}
$this->info('Processed ' . count($users) . ' users.');
});
$this->info('All users processed successfully!');
// You can also chunk by ID (recommended for very large tables)
// User::chunkById(500, function ($users) {
// foreach ($users as $user) {
// // Process user
// }
// }, $column = 'id', $alias = null);
}
}
How it works: This snippet illustrates how to use Eloquent's `chunk()` and `chunkById()` methods to process large datasets efficiently. Instead of retrieving all records at once into memory, `chunk()` retrieves them in smaller batches (e.g., 1000 records at a time) and passes each batch to a given callback. This significantly reduces memory consumption, making it ideal for console commands, data migrations, or other long-running processes that deal with many database records. `chunkById()` is similar but ensures consistent ordering and efficient pagination for even larger tables.