Creating Drush ready batches with Drupal Batch API in Drupal 8/9

When certain operations take more time to execute than the normal PHP script, we tend to break them into smaller batches which can be executed by a cron job to avoid PHP execution timeout errors and have a more balanced distribution of the server resources.

Drupal comes with an excellent Batch API which uses queue workers to simplify this for us. An example in this guide is done with the updated version of Batch API using the \Drupal\Core\Batch\BatchBuilder, which is basically just an object-oriented version of previously procedural style Batch API. BatchBuilder was introduced with the Drupal core version 8.6.

The objective of this short guide is the following:

• create service for creating batches using Drupal Batch API
• create a Drupal form for running the batches
• create a Drush command for running the batches

The end result will be a form for running batch processing as shown below (Image 1).

As a bonus, we also create a Drush command which is used for running batch processing. This becomes really useful when batch processing should be executed on a scheduled basis using cron.

For demonstration purposes, the service will generate an array of strings to be processed as "nodes". But you get the idea, instead of having a list of strings, those can be real objects ready for batch processing. I will be using the placeholder namespace called modulename which can be renamed to whatever is needed.

First, we need to register batch service inside the modulename.services.yml file.

Filename: modulename.services.yml

services:
  modulename.batch:
    class: 'Drupal\modulename\BatchService'
    arguments:
      - '@logger.factory'

Next, we add batch service files (interface and the batch service file itself).

Filename: src/BatchServiceInterface.php

<?php

namespace Drupal\modulename;

/**
 * Defines batch service interface.
 */
interface BatchServiceInterface
{

  /**
   * Create batch.
   *
   * @param int $batchSize
   *   Batch size.
   */
    public function create(int $batchSize): void;

    /**
     * Batch operation callback.
     *
     * @param array $batch
     *   Information about batch (items, size, total, ...).
     * @param array $context
     *   Batch context.
     */
    public static function process(array $batch, array &$context): void;

    /**
     * Bach operations 'finished' callback.
     *
     * @param bool $success
     *   TRUE if processing was successfully completed.
     * @param mixed $results
     *   Additional data passed from $context['results'].
     * @param array $operations
     *   List of operations that did not complete.
     */
    public static function finishProcess($success, $results, array $operations): void;
}

Filename: src/BatchService.php

<?php

namespace Drupal\modulename;

use Drupal\Core\Logger\LoggerChannelFactoryInterface;
use Drupal\Core\Batch\BatchBuilder;
use Drupal\Core\StringTranslation\StringTranslationTrait;

/**
 * Service for creating batches.
 */
class BatchService implements BatchServiceInterface
{
    use StringTranslationTrait;

    /**
     * The logger channel.
     *
     * @var \Drupal\Core\Logger\LoggerChannelInterface
     */
    protected $loggerChannel;

    /**
     * Constructor.
     *
     * @param \Drupal\Core\Logger\LoggerChannelFactoryInterface $loggerFactory
     *   The logger factory.
     */
    public function __construct(LoggerChannelFactoryInterface $loggerFactory)
    {
        $this->loggerChannel = $loggerFactory->get('<modulename>');
    }

    /**
     * {@inheritdoc}
     */
    public function create(int $batchSize = 10): void
    {
        /** @var \Drupal\Core\Batch\BatchBuilder $batchBuilder */
        $batchBuilder = (new BatchBuilder())
            ->setTitle($this->t('Running node updates...'))
            ->setFinishCallback([self::class, 'finishProcess'])
            ->setInitMessage('The initialization message (optional)')
            ->setProgressMessage('Completed @current of @total. See other placeholders.');

        $nodes = array_fill(0, 13, 'test');
        $total = count($nodes);
        $itemsToProcess = [];
        $i = 0;
        // Create multiple batch operations based on the $batchSize.
        foreach ($nodes as $node) {
            $i++;
            $itemsToProcess[] = $node;
            if ($i == $total || !($i % $batchSize)) {
                $batchBuilder->addOperation([BatchService::class, 'process'], [
                    'batch' => [
                        'items' => $itemsToProcess,
                        'size' => $batchSize,
                        'total' => $total,
                    ],
                ]);
                $itemsToProcess = [];
            }
        }

        batch_set($batchBuilder->toArray());
        $this->loggerChannel->notice('Batch created.');
        if (PHP_SAPI === 'cli' && function_exists('drush_backend_batch_process')) {
            drush_backend_batch_process();
        }
    }

    /**
     * {@inheritdoc}
     */
    public static function process($batch, &$context): void
    {
        // Process elements stored in the each batch (operation).
        foreach ($batch['items'] as $item) {
            $context['results'][] = $item;
            sleep(1);
        }
        // Message displayed above the progress bar or in the CLI.
        $processedItems = !empty($context['results']) ? count($context['results']) : $batch['size'];
        $context['message'] = 'Processed ' . $processedItems . '/' . $batch['total'];

        \Drupal::logger('<modulename>')->info(
            'Batch processing completed: ' . $processedItems . '/' . $batch['total']
        );
    }

    /**
     * {@inheritdoc}
     */
    public static function finishProcess($success, $results, array $operations): void
    {
        // Do something when processing is finished.
        if ($success) {
            \Drupal::logger('<modulename>')->info('Batch processing completed.');
        }
        if (!empty($operations)) {
            \Drupal::logger('<modulename>')->error('Batch processing failed: ' . implode(', ', $operations));
        }
    }
}

Code for preparing batches is now ready. To make use of it we need a Drupal form and/or Drush command to execute it. Our form will be available in the administration area on the following path:   /admin/config/batch-form.

Filename: modulename.routing.yml

modulename.batch_form:
  path: '/admin/config/batch-form'
  defaults:
    _form: 'Drupal\modulename\Form\BatchForm'
    _title: 'Batch form'
  requirements:
    _permission: 'administer site configuration'
  options:
    _admin_route: TRUE

Filename: src/Form/BatchForm.php

<?php

namespace Drupal\modulename\Form;

use Drupal\modulename\BatchServiceInterface;
use Drupal\Core\Form\FormBase;
use Drupal\Core\Form\FormStateInterface;
use Drupal\Core\Messenger\MessengerInterface;
use Symfony\Component\DependencyInjection\ContainerInterface;

/**
 * Defines batch form.
 */
class BatchForm extends FormBase
{

  /**
   * The messenger.
   *
   * @var \Drupal\Core\Messenger\MessengerInterface
   */
    protected $messenger;

    /**
     * The batch service.
     *
     * @var \Drupal\modulename\BatchServiceInterface
     */
    protected $batch;

    /**
     * Constructor.
     *
     * @param \Drupal\Core\Messenger\MessengerInterface $messenger
     *   The messenger.
     * @param \Drupal\modulename\BatchServiceInterface $batch
     *   THe batch service.
     */
    public function __construct(MessengerInterface $messenger, BatchServiceInterface $batch)
    {
        $this->messenger = $messenger;
        $this->batch = $batch;
    }

    /**
     * {@inheritdoc}
     */
    public static function create(ContainerInterface $container)
    {
        return new static(
            $container->get('messenger'),
            $container->get('modulename.batch'),
        );
    }

    /**
     * {@inheritdoc}
     */
    public function getFormId()
    {
        return 'test_batch_form';
    }

    /**
     * {@inheritdoc}
     */
    public function buildForm(array $form, FormStateInterface $form_state)
    {
        $form['actions']['#type'] = 'actions';
        $form['descriptions'] = ['#markup' => '<p>This form will run batch processing.</p>'];
        $form['actions']['submit'] = [
            '#type' => 'submit',
            '#value' => $this->t('Run batch process'),
            '#button_type' => 'primary',
        ];
        return $form;
    }

    /**
     * {@inheritdoc}
     */
    public function submitForm(array &$form, FormStateInterface $form_state)
    {
        $this->batch->create();
    }
}

And for our last step we will register a new Drush command which will be used for running batch processing from CLI. Command also has the option to specify the batch size (something that form currently does not support). The default batch size is 10.

Filename: drush.services.yml

modulename.batch_command:
    class: Drupal\modulename\Commands\BatchCommand
    arguments:
        - '@modulename.batch'
    tags:
        - { name: drush.command }

Filename: src/Command/BatchCommand.php

<?php

namespace Drupal\modulename\Commands;

use Drupal\modulename\BatchServiceInterface;
use Drush\Commands\DrushCommands;

/**
 * Defines drush command.
 */
class BatchCommand extends DrushCommands
{

  /**
   * The batch service.
   *
   * @var \Drupal\modulename\BatchServiceInterface
   */
    protected $batch;

    /**
     * Constructor.
     *
     * @param \Drupal\modulename\BatchServiceInterface $batch
     *   The batch service.
     */
    public function __construct(BatchServiceInterface $batch)
    {
        $this->batch = $batch;
    }

    /**
     * Run batch command.
     *
     * @param array $options
     *   Command options.
     *
     * @command modulename-batch:run
     * @aliases modulename-br
     * @option batch Batch size. Default: 10
     * @usage modulename-batch:run
     */
    public function run(array $options = ['batch' => 10])
    {
        $this->batch->create($options['batch']);
    }
}

That is it. We now can run our Drush command to start batch processing with a batch size of 10. Image 2 shows the output from running batch operations by using the Drush command:

drush modulename-batch:run --batch=10