Introduction

While certainly not a new concept, Queues have become a powerful and handy feature of modern frameworks. They enable our normally thread-locked apps to asynchronously dispatch tasks, allowing workflows and, especially, users to continue forward. Expectations are high for websites to deliver fast and get content on the page to maintain a good user experience. Laravel is a well-tuned PHP framework, and leveraging the Queue feature is a snap. I won’t get into configuring Laravel Queues here, as what I really want to explore is handling errors in the Jobs executed in the Queue.

One of the challenges of using queues, and this is not Laravel-specific, is that errors can be silent or disappear deep down into backtraces. A bad API key or temporary service outage can cause a lot of headaches for Jobs, and depending on how error-handling has been set up, those failures can produce some serious issues. One may have a SendMail Job that keeps hitting a service improperly, resulting in an error. Too many attempts, and the service may decide to block one’s application, or worse, generate more charges for exceeding bandwidth. Laravel’s documentation for Queues is very good and defines many options available for handling errors and problems.

Handling Errors

First, let’s take a quick look at how Laravel handles errors when running Jobs in the Queue. When an error occurs in a Job, it will automatically be released back into the Queue and will be processed again. The number of times the Job will be processed depends on the maximum number of tries defined by the application or in the –tries option of the queue:work Artisan command. Additionally, a Job can specify its own (among other configurations) maximum tries and timeout values.

If you want to handle the tries dynamically, you can, instead of adding the $tries property, the following public method may replace it in the Job class:

In addition to setting the $timeout value, a $failOnTimeout = true property may be added, meaning that if it times out, it will fail and will not be retried. In addition to this, the following method can be used to retry the Job but only up until a specific amount of time:

In addition to these methods, an especially useful one is backoff(). This allows you to define a strategy as part of the retries:

This means that the first retry will delay 3 seconds, the second will be 10 seconds, and finally, the third will be 20 seconds. When working with an API that may be notorious for not handling stress well, you can use this to let it ‘breathe’ for a moment.

Not All Errors Are Equal

There are a variety of errors that can occur when processing a Job, but they boil down to just a couple:

1. Temporary — Outages, rate limiting, the dreaded database deadlock.
2. Permanent — Missing assets, invalid data, etc., meaning that retrying will not help.

Temporary errors are well-handled using tries, while a permanent error can saturate the Queue. How do we handle that? If you wrap your code in a try/catch, you can decide at that point with something like this:

In this case, a rate-limited exception will release the Job back into the Queue, but after a delay of 60 seconds. This depends on the API’s thresholds, of course. If the connection fails due to a network outage or bad credentials, the error can be handled by triggering the Job to fail. Instead of passing the Exception to the fail() method, you may also pass a string description.

The fail() method is especially helpful before an exception is triggered. If, in your code in the handle() method finds that the data is not correct, or that the process, for whatever reason, should not continue. Calling fail() can stop it altogether. Be sure to pass the description of the issue to the fail method, too.

Middleware can be used in a Job to define which exceptions will trigger a failure instead of retrying by adding the following method to the class:

Afterthought

Be sure to visit the Laravel documentation on Queues and read up on error handling. It will save you from massive amounts of headaches later.  Because the Laravel community is so large and active, you should be able to find many solutions to help with handling errors, especially depending on the infrastructure selected to run your Queue (Redis, RabbitMQ, etc.). Take a look at Horizon, too, for providing a dashboard to monitor your Job Queue, in addition to some great features like supporting auto-balancing, metrics, and notifications.