Message Queues and Delivery Guarantees

Why put a queue in between

When a customer places an order, the checkout must save the order and take payment. Sending the confirmation email, updating analytics, notifying the warehouse and generating an invoice PDF can all happen a few seconds later. If checkout calls each of them directly, it is only as fast and as reliable as the slowest one.

• Faster responses: the user does not wait for slow follow-up work.
• Resilience: if the email service is down, messages wait in the queue instead of failing checkout.
• Load levelling: a burst of 10,000 orders is absorbed by the queue and processed at a steady rate.
• Scaling: add more consumers to drain the queue faster (the competing consumers pattern).

Acknowledgements and visibility timeouts

A consumer that takes a message might crash before finishing it. So queues do not delete a message when it is handed out. Instead it becomes invisible for a while (the visibility timeout, in Amazon SQS terms) or is marked unacknowledged (RabbitMQ). The consumer acknowledges it after processing; only then is it deleted. If no acknowledgement arrives in time, the message is delivered again.

import java.util.*;

class VisibilityQueue {
    private final double timeoutSeconds;
    private final Deque<String> ready = new ArrayDeque<>();
    private final Map<String, Double> inFlight = new LinkedHashMap<>(); // id -> visible again at
    private final Map<String, String> bodies = new HashMap<>();
    private int nextId = 1;

    VisibilityQueue(double timeoutSeconds) {
        this.timeoutSeconds = timeoutSeconds;
    }

    synchronized String send(String body) {
        String id = "m" + nextId++;
        bodies.put(id, body);
        ready.addLast(id);
        return id;
    }

    // Returns {id, body}, or null if nothing is available.
    synchronized String[] receive(double now) {
        Iterator<Map.Entry<String, Double>> it = inFlight.entrySet().iterator();
        while (it.hasNext()) {
            Map.Entry<String, Double> e = it.next();
            if (now >= e.getValue()) { // timed out: make it visible again
                ready.addLast(e.getKey());
                it.remove();
            }
        }
        String id = ready.pollFirst();
        if (id == null) {
            return null;
        }
        inFlight.put(id, now + timeoutSeconds);
        return new String[] { id, bodies.get(id) };
    }

    synchronized void ack(String id) {
        if (inFlight.remove(id) != null) {
            bodies.remove(id);
        }
    }
}

from collections import deque

class VisibilityQueue:
    def __init__(self, timeout_seconds):
        self.timeout = timeout_seconds
        self.ready = deque()
        self.in_flight = {}  # id -> time it becomes visible again
        self.bodies = {}
        self.next_id = 1

    def send(self, body):
        msg_id = f"m{self.next_id}"
        self.next_id += 1
        self.bodies[msg_id] = body
        self.ready.append(msg_id)
        return msg_id

    def receive(self, now):
        """Returns (id, body), or None if nothing is available."""
        for msg_id, visible_at in list(self.in_flight.items()):
            if now >= visible_at:  # timed out: make it visible again
                del self.in_flight[msg_id]
                self.ready.append(msg_id)
        if not self.ready:
            return None
        msg_id = self.ready.popleft()
        self.in_flight[msg_id] = now + self.timeout
        return msg_id, self.bodies[msg_id]

    def ack(self, msg_id):
        if self.in_flight.pop(msg_id, None) is not None:
            del self.bodies[msg_id]

Delivery guarantees

Why duplicates are unavoidable with at-least-once: the consumer processes the message, then crashes (or the network drops) before its acknowledgement arrives. The queue cannot tell this apart from a consumer that never processed it, so it redelivers.

Messages that fail again and again (a malformed payload, a bug) should not loop forever. After N attempts, move them to a dead-letter queue where a person or a tool can inspect them.

Ordering

With several consumers working in parallel, messages finish out of order even if the queue hands them out in order. If order matters only per entity (all events of order 1042 in sequence), route each entity's messages to one partition or message group, processed by one consumer at a time. Global ordering across everything does not scale and is rarely needed.

Common mistakes

• Assuming each message is delivered exactly once, and charging or emailing twice on redelivery.
• Acknowledging before processing, then losing the message when the consumer crashes.
• A visibility timeout shorter than processing time, so messages are processed in parallel twice.
• No dead-letter queue, so one poison message is retried forever.

Interview questions

Try answering out loud before you open each one.

Check yourself

Related topics

• Idempotent OperationsApplication Layer
• Task Queues and Background JobsAsynchronism
• Pub/Sub and Event StreamingAsynchronism

Further reading

• Amazon SQS visibility timeout AWS documentation
• Reliability guide: acknowledgements and confirms RabbitMQ documentation
• Competing Consumers pattern Microsoft Azure Architecture Center