Queues and jobs

Sending an email, processing an image, exporting a report take seconds. If that happens during the request, the user watches a spinner the whole time. A queue defers the work: the request answers right away, and the job runs in the background.

How it works

The application puts a job into a queue — a database table or Redis. A separate process, the worker, keeps taking jobs from it and running them. The QUEUE_CONNECTION variable in .env sets where the queue is stored.

Creating a job

php artisan make:job ProcessPhoto

The work is described in the handle() method, and the data is passed through the constructor:

namespace App\Jobs;
 
use App\Models\Photo;
use Illuminate\Contracts\Queue\ShouldQueue;
use Illuminate\Foundation\Queue\Queueable;
 
class ProcessPhoto implements ShouldQueue
{
    use Queueable;
 
    public function __construct(public Photo $photo) {}
 
    public function handle(): void
    {
        // shrink the image, create a thumbnail…
        $this->photo->update(['processed' => true]);
    }
}

The ShouldQueue interface is what means "run in the background".

Dispatching to the queue

use App\Jobs\ProcessPhoto;
 
public function store(Request $request)
{
    $photo = Photo::create([...]);
 
    ProcessPhoto::dispatch($photo);
 
    return back()->with('status', 'The photo is uploaded and being processed');
}

The response goes to the user immediately — processing starts when the job's turn comes.

Delayed start

ProcessPhoto::dispatch($photo)->delay(now()->addMinutes(5));
 
ProcessPhoto::dispatch($photo)->onQueue('images');

onQueue() puts the job into a separate queue — so urgent tasks do not wait behind slow ones.

Running the worker

Nobody runs the jobs until a worker is started:

php artisan queue:work

On a production server this process must run all the time. A process manager — Supervisor or systemd — watches it and restarts the worker if it stops.

Retries

A job can fail: the mail server is down, the network dropped. The number of attempts and the pause between them are set with properties:

class ProcessPhoto implements ShouldQueue
{
    use Queueable;
 
    public int $tries = 3;
    public int $backoff = 60;
 
    public function failed(\Throwable $exception): void
    {
        // notify the administrator
    }
}

Jobs that ran out of attempts go into the failed jobs table:

php artisan queue:failed
php artisan queue:retry all

Developing without a worker

The value QUEUE_CONNECTION=sync runs jobs immediately, during the request. That is handy for development and tests, but not for a live site.

Common mistake

A worker loads the code once at start-up. After the site is updated it keeps running jobs with the old code. Restart the workers on every deployment:

php artisan queue:restart