Queues are a natural phenomenon in everyday life. You queue to buy food, to pay in the supermarket, or to get on a plane. But queues are also used in software to handle heavy processes.
A queue can help devs like me and you store a request for later processing. It's a common practice to use a queue to handle data exports from a site, for example.
What you'll learn
In this post I'll compare queues in software development with queues in a fast food restaurant. You'll learn:
- What's a queue
- Uses of a queue
- Caveats of queues
- Benefits of using a queue
- Common services you can use
Queuing for food: First In, First Out (FIFO)
When you go to a fast food place you need to first get in line to place your order.
You wait in line until it's your turn to say whatever you're gonna have. Ideally, no one gets in the line; that would be disrespectful and will cause some complaints.
This queue operates with a FIFO principle. That means that the first person dispatched is the first to be in line. The first that gets in line is the first to get out.
You can picture each person in line as a heavy processing job request from a client to the application. An asynchronous worker processes each of these requests that are waiting in line. And by worker I mean something like a server that's not reachable by your customers.
Each of these requests should respond in the order in which they were issued. But it's not always the case.
Preparing your food: Order might change
Once you place your order, the cashier sends it to the cooks, so they can start preparing it. The orders get into another queue.
Now, imagine there's only one cooker. You ordered a triple cheeseburger with extra onions, a vanilla milkshake, and small fries. The person after you only ordered some small fries.
The cooker might say: let me serve the customer that only ordered fries first because that's quick. That puts you after that customer in the queue even though you ordered first.
In software development, queues are usually used with distributed systems. And in these type of systems the order of the elements in the queue might change.
There are special types of queues that guarantee the order, but not every queue does it. So your application must not depend on the order of the requests.
There's another problem you need to manage. What if there's more than one cooker and two or more of them start preparing the same order? It's very common for an application to have multiple workers that read the same queue.
Mistakes in food preparation: Handling idempotency
If two cooks take your order they will each prepare you what you asked for. You'll end up with two cheeseburgers instead of one. There should be a way to avoid two cooks working on the same order.
A caveat of a distributed queue is that two workers can read the same message at the same time. So we say these requests should be idempotent. That is, the same operations requested multiple times should produce the same result.
For example, a DELETE request on a REST API should delete a resource the first time it's called. This usually returns a 200 (OK) or 204 (No Content) response. The result should be the same if that same request is called a second time: the deletion of the resource.
Some systems only disable the resource instead of deleting it from the database. In that case, the result (side effect in the application) and the response (literal response from the API) will be the same.
If the system deleted the resource the first time, then you'll receive a 404 (Not found) response on the second call. Here two identical requests had the same result, but a different response.
In both cases the delete request was idempotent. Two same requests resulted in the same outcome.
Getting your food: Notifying the end user
It's common to receive a device that vibrates and lights on when your order is ready. That's how the restaurant notifies you that your order is ready for pickup.
Workers also need a way to communicate to its users. Email is a great solution since most of the processing workers do is asynchronous.
The only job of the queue is to receive and store requests until they're processed.
Why do we need queues?
Queues improve user experience
You don't want to wait in front of the cashier until your order is ready; you don't know how long will it take.
Similarly, you don't want your users to wait while looking at a loading spinner. Moreover, it can be taxing on the client-facing servers.
The HTTP request window is short. Applications close the connection between server and client when a request is taking more than 30-60 seconds, resulting in a timeout.
How can you handle a resource-intensive task that takes more than 60 seconds? You send it to a queue, and a worker in the background can take that message and process it.
Queues are language agnostic
It doesn't matter if you send a message to a queue with Python and receive it with JavaScript. The intermediary, called the message broker, will handle it.
Queues decouple your systems
With queues, you can separate concerns and allow a different service or part of your application to handle specific requests.
The messages in the queue will be waiting even if your workers fail.
Popular queue services and message brokers
You don't have to implement the queue's internal logic, it's a solved problem. You can instead use one of these popular services:
I've only used Amazon SQS, and it's fast to start working with it. It gets complicated as you dig deeper, but it's a great way to start.
What I didn't tell you
There are other factors to consider when working with queues and message brokers. The intention of this post is to only give a general overview of how the mechanism works.
To give you an idea, you need to configure and/or handle, among many others:
- Retry policy
- Failed messages
- Security of your queues
- Duration of messages in the queue
- Reliability and availability of your queues and message brokers
Wrap up
Queues are a great addition to your software developer arsenal. If you need to do a resource-intensive task, a queue should be one of your options.
Thanks for reading me! 👋
**Cover image taken from unDraw
