- Accept
The API accepts the client’s job and returns 202 Accepted with a job ID. Acceptance is not completion.
- Queue
The job waits in a queue. Queue depth and admission policy control back pressure.
- Worker
A worker consumes the job and writes the result to storage.
- Status
The client later reads job status or the stored result.
Slow work runs outside the immediate request path.
How to read this model
A submission terminal beside a waiting magazine and worker fixture. The object is a teaching analogy. Inspecting a part highlights its role; it does not execute or simulate the software. Cabinet trays are records, workpieces are data, and latches are checks. Unless an explanation states a quantity, compartments and sample workpieces do not denote data counts. Part dimensions are not a quantitative scale.
Concept redrawn from Introduction to architecting systems for scale.
Asynchronous workflows shorten user-facing requests by moving expensive work out of line. They can also compute results in advance, such as periodic aggregates. The trade-off is a delayed result and a state machine for pending, failed, and completed work.
Message queues
A message queue receives, holds, and delivers messages. A typical job flow is:
- The application publishes a job and returns a job ID or status to the client.
- A worker claims the job, performs the work, and records the result.
- The client reads the status later, or the system sends a completion event.
client -> API -> queue -> worker -> database
^ | |
| +-- 202 Accepted -------+
+---------- GET /jobs/{id} -----+
The user is not blocked by the worker. A social post can appear in the author's timeline immediately while fan-out to every follower happens later. The interface must say whether "accepted" means stored, queued, or complete.
- Accept
The API accepts the client’s job and returns 202 Accepted with a job ID. Acceptance is not completion.
- Queue
The job waits in a queue. Queue depth and admission policy control back pressure.
- Worker
A worker consumes the job and writes the result to storage.
- Status
The client later reads job status or the stored result.
A 202 response supplies a job ID; it does not assert that the job succeeded.
How to read this model
A job feeder, work fixture, and completed-record tray. The object is a teaching analogy. Inspecting a part highlights its role; it does not execute or simulate the software. Cabinet trays are records, workpieces are data, and latches are checks. Unless an explanation states a quantity, compartments and sample workpieces do not denote data counts. Part dimensions are not a quantitative scale.
Redis can act as a simple broker, but messages can be lost depending on configuration. RabbitMQ supports AMQP and requires node operations. Amazon SQS is hosted, but delivery can be delayed and a message can be delivered more than once. Make consumers idempotent and acknowledge only after durable progress.
Task queues
A task queue carries a task and its related data, runs it, and can return a result. Task queues support scheduled or computationally intensive work. Celery is a common Python-oriented task queue with scheduling support.
Back pressure
Little's law relates the average number of items in a system to arrival rate and time in the system:
items in system = arrival rate * time in system
If producers add work faster than workers finish it, the queue grows. An unbounded queue can exhaust memory, force disk reads, and make every job slower. Apply back pressure by limiting queue depth or admission rate. Once the queue is full, return a stable busy response such as HTTP 503 and let clients retry later with exponential backoff.
Monitor queue depth, oldest message age, processing time, retry count, and dead-letter volume. These metrics tell you whether more workers will help or whether the producer must slow down.
When synchronous work is better
An inexpensive calculation or a real-time interaction may cost more to queue than to execute. Queues add serialization, storage, retries, and a delayed result. Use them when the work is slow, bursty, independently retryable, or can finish after the user response.
See latency and throughput and communication protocols for the request-time trade-offs.