mattgrayisok

Craft CMS has an acceptable method of executing background tasks that works ok in the majority of situations. However there's one significant caveat which I described in Craft in Docker Part 6 - Performance:

When a user logs into the control panel some JS will make an ajax request to the back end which is essentially used as a trigger to run whatever task is at the head of the background tasks queue. This ajax request fires on every control panel page load and, because it's an ajax request whose response can be ignored, it doesn't impact the user's overall experience.

This seems sensible until you run into the specific issue that I have discovered a couple of times now. These asynchronous background tasks, initiated by ajax requests, consume a PHP-FPM worker process for the length of time that they're running. If a user visits several control panel pages this can cause multiple tasks to begin running simultaneously, each consuming a PHP-FPM worker process. The default maximum number of simultaneous PHP-FPM worker processes in the official PHP docker container is set to a somewhat conservative 5. So if we end up in a situation where 5 slow background tasks have all started running - each consuming a worker process - PHP will start refusing to serve any more requests, including legitimate requests from end users who are just trying to visit our website!

In that article I suggested a quick fix which was to simply increase the number of PHP child workers to prevent them all being used by background tasks. This fix isn't really a solution, it just kicks the can down the road a little. It would be much better if the tasks were executed using a completely separate PHP master process, isolating them from the process which is handling normal user requests. This can be achieved by disabling Craft's automatic-queue-processing-via-ajax functionality in config/general.php:

return [
    ...
    'runQueueAutomatically' => false,
    ...
];

And instead, processing the queue from PHP on the command line using Craft's CLI script:

./craft queue/run

We just need to make this run on a regular basis. The simplest solution to make that happen would be to use a cron job.

As we're running Craft in Docker it's tempting to try to get this cron job to run inside our PHP container, treating the container like a regular multi-tasking server, however docker containers are designed specifically to only execute a single main process - PHP-FPM in our case. To run a cron scheduler alongside this we'd need to get it installed into our container and then set the main process of the container to launch both the scheduler and our PHP-FPM process. This is possible, but not advised as it makes it difficult for docker to cleanly monitor and exit the container to handle main process exceptions.

There are two, better alternatives.

docker-compose exec

We can create a cron job on the host which executes the CLI command using docker-compose exec. This is a bit of a half solution as it doesn't require running multiple main processes in your PHP-FPM container but does run the queue process inside the same container so we haven't fully separated the two concerns.

Assuming our php container is named php:

#Crontab

* * * * * docker-compose exec php ./craft queue/run

docker-compose run

If we want to separate the background tasks from user requests completely we can run them in a different container, launched via a cron defined on the host:

#Crontab

* * * * * docker-compose run --rm php ./craft queue/run

Rather than run in our primary PHP-FPM container this will create a new container specifically to execute the background tasks and then clean it up once it has finished. This provides complete isolation as the processes in the different containers aren't able to influence each other 👌

There are a couple of minor downsides to this approach though:

• It takes a little longer to start up and shut down because it's creating and cleaning up a new container.
• It requires a little more memory as it won't be able to share any in-memory libraries that the primary container already has loaded.

Codified Cron Jobs

With the previous two solutions we've had to rely on our dev-ops guy to add the cron jobs to the host manually. But the cron jobs are an indispensable part of our Craft project so really we should codify their parameters and add them to our project repo so they don't get lost.

We can achieve this using another docker container into which we can mount a config file containing our cronjob definitions. We need to make two changes to our standard Craft in Docker docker-compose.yml.

First add a new container:

cronjobs:
    privileged: true
    image: mcuadros/ofelia:latest
    volumes:
        - /var/run/docker.sock:/var/run/docker.sock
        - ./cronjobs.ini:/etc/ofelia/config.ini

This is using the image mcuadros/ofelia which is a task runner specifically designed to execute tasks by running docker containers. We are mounting the host's docker socket into the container so that it is able to create additional containers on the host on our behalf.

Second, give our PHP-FPM container a specifically defined container name:

php:
    container_name: craft-in-docker-php
    ...

Finally, create a cronjob config file in the root of our project which is mounted into our new scheduling container. I've called mine cronjobs.ini:

[job-exec "background-tasks"]
schedule = 0 * * * * *
container = craft-in-docker-php
command = ./craft queue/run

We've specified a schedule (with per-second precision unlike cron), the name of the container in which to execute the command and the command itself.

Try it out with docker-compose up. Now all of our cron jobs and the method for executing them is codified and contained in our project repo.

You might have noticed that in this final setup we're using Ofelia's job-exec option which is the equivalent of running docker-compose exec. I also mentioned earlier that this was only a half solution because it doesn't provide clean separation between our user facing PHP-FPM container and our background tasks. This is still true in this new configuration. Using Ofelia's job-run option would be better but I ran into issues when try to get Ofelia to pull images from a private docker registry so settled on this solution for now.

You can find a branch in the main Craft in Docker repo which demonstrates this setup.

Long Running Queue Runner

Thanks to @dsmrt for reminding me about this.

If we want our tasks to process immediately, we don't have any other schedule based tasks to worry about, and we don't mind sacrificing some server resources to do it, we also have the option to run a copy of our application in a different container whose sole responsibility it is to run pending background tasks. We've demonstrated using docker-compose exec that our existing Craft project image is able to perform this work so we simply need to make a copy of that container with a custom command.

If you're using something similar to the setup described in my main Craft in Docker series you'll need to add the following to docker-compose.yml:

background-tasks:
    build:
        context: .
        dockerfile: ./docker-config/php/Dockerfile
    expose:
        - 9000
    volumes:
        - cpresources:/var/www/html/web/cpresources
        - ./src/composer.json:/var/www/html/composer.json
        - ./src/composer.lock:/var/www/html/composer.lock
        - ./src/config:/var/www/html/config
        - ./src/modules:/var/www/html/modules
        - ./src/templates:/var/www/html/templates
        - ./src/web:/var/www/html/web
    environment:
        ENVIRONMENT: dev
        DB_DRIVER: mysql
        DB_SERVER: database
        DB_USER: project
        DB_PASSWORD: project
        DB_DATABASE: project
        DB_SCHEMA: public
        DB_TABLE_PREFIX: craft_
        SITE_URL: http://localhost
        SECURITY_KEY: AAAAAAAAAAAAAAAAAAAAAAAAAAA
    command: ./craft queue/listen 10

Tip: Instead of duplicating lots of lines in your docker-compose you can use YAML aliases to neaten things up.

This is just an exact copy of your existing PHP-FPM container definition but with a custom command added on the end. You'll also need to add something similar to your production docker-compose.yml but with less mounts and a restart policy.

Now, whenever you run docker-compose up a container will be created whose sole responsibility it is to process your background tasks. I have set this one up to poll for jobs every 10 seconds but you can adjust to your preference.

This is the strategy that I have always used for Laravel projects in the past and works well because it allows you to easily stop and restart the processing of background tasks if they're causing any problems. Keep in mind however that this new container will be holding a copy of Craft, Yii and any applicable dependencies in memory and you'll need to watch out for memory leaks due to its long-running nature.