Cron and queues

Cron is the Drupal scheduler: it periodically runs the housekeeping of all modules — clean-up, update checks, search indexing. A queue is a list of deferred tasks that are done bit by bit in the background. Together they let you avoid doing long work while a visitor waits for a page.

A scheduled task

A module adds its work to the scheduler with the hook_cron hook in the blog.module file:

// web/modules/custom/blog/blog.module
 
/**
 * Implements hook_cron().
 */
function blog_cron(): void
{
    \Drupal::database()->delete('blog_subscribers')
        ->condition('created', strtotime('-2 years'), '<')
        ->execute();
 
    \Drupal::logger('blog')->notice('Old subscriptions deleted.');
}

On every scheduler run the function deletes subscriptions older than two years and writes a line to the log.

Running

vendor/bin/drush cr
vendor/bin/drush cron
vendor/bin/drush ws --count=5

The second command runs the scheduler by hand, the third shows the latest log entries — "Old subscriptions deleted." is among them.

On a production server the scheduler is started by the operating system schedule, for example every 15 minutes:

*/15 * * * * cd /var/www/site && vendor/bin/drush cron

The line is added to the server job table with the crontab -e command. Without it Drupal runs cron itself, together with visitor requests, once every three hours — this is set at Configuration → System → Cron.

Why a queue

The blog_cron() hook must be fast: while it runs, the other modules wait. If thousands of records have to be processed — for example notifying every subscriber about a new article — the tasks are put into a queue, and the scheduler works through it in portions.

Adding to a queue

Let us put a task into the queue when an article is created:

// web/modules/custom/blog/blog.module
 
use Drupal\node\NodeInterface;
 
/**
 * Implements hook_ENTITY_TYPE_insert() for node.
 */
function blog_node_insert(NodeInterface $node): void
{
    if ($node->bundle() !== 'article') {
        return;
    }
 
    $emails = \Drupal::database()->select('blog_subscribers', 's')
        ->fields('s', ['email'])
        ->execute()
        ->fetchCol();
 
    $queue = \Drupal::queue('blog_notify');
 
    foreach ($emails as $email) {
        $queue->createItem(['email' => $email, 'nid' => $node->id()]);
    }
}

\Drupal::queue('blog_notify') returns the queue with this name, createItem() puts one task into it — any array of data. Saving the article stays fast: no mail is sent here.

The worker

Tasks are done by a QueueWorker plugin. Its folder is src/Plugin/QueueWorker, and the id equals the queue name:

// web/modules/custom/blog/src/Plugin/QueueWorker/NotifyWorker.php
namespace Drupal\blog\Plugin\QueueWorker;
 
use Drupal\Core\Queue\Attribute\QueueWorker;
use Drupal\Core\Queue\QueueWorkerBase;
use Drupal\Core\StringTranslation\TranslatableMarkup;
 
#[QueueWorker(
    id: 'blog_notify',
    title: new TranslatableMarkup('Subscriber notifications'),
    cron: ['time' => 30],
)]
class NotifyWorker extends QueueWorkerBase
{
    public function processItem($data): void
    {
        \Drupal::logger('blog')->notice('Notification for @email about article @nid.', [
            '@email' => $data['email'],
            '@nid' => $data['nid'],
        ]);
    }
}

processItem() receives one task. Here it only writes to the log — sending the mail would go in this place. The cron: ['time' => 30] entry lets the scheduler spend up to 30 seconds per run on this queue; the remaining tasks wait for the next run.

Checking

vendor/bin/drush cr
vendor/bin/drush queue:list
vendor/bin/drush queue:run blog_notify

Create an article, then run the commands. queue:list shows the blog_notify queue and the number of tasks in it, queue:run executes them at once without waiting for cron.

Common mistake

If processItem() throws an exception, the task stays in the queue and is retried on the next run. That helps with a temporary failure, but a task with a permanent error — for example a non-existent address — is retried forever. Catch expected errors inside the worker and log them, and keep the exception for the cases where a retry really helps.