A thread is an independent path of execution managed by the operating system. Each thread has a stack for function calls and local variables, and an instruction pointer that records where execution resumes.
- Task queue
Pending tasks wait until a worker pulls them.
- Worker pool
Worker threads 1, 2, and 3 represent the illustrated process pool.
- Shared memory
Workers in a process can share memory. Coordinate conflicting reads and writes.
Pool size limits active execution. Shared memory still needs synchronization.
How to read this model
A shared job magazine with three worker stations. The object is a teaching analogy. Inspecting a part highlights its role; it does not execute or simulate the software. Cabinet trays are records, workpieces are data, and latches are checks. Unless an explanation states a quantity, compartments and sample workpieces do not denote data counts. Part dimensions are not a quantitative scale.
The OS scheduler preempts threads and gives each a time slice. When a thread performs blocking I/O, the OS marks it blocked and schedules another runnable thread. When the I/O completes, the thread becomes runnable and waits for another slice.
running -> hits I/O -> blocked -> I/O completes -> runnable
-> scheduler selects it -> running
Preemption means the OS can pause a thread before its function finishes. A context switch saves the current registers and state, loads another thread's state, and updates scheduler bookkeeping.
Threads share a process heap
Threads in one process share the heap and global variables. Shared memory is fast because threads do not serialize every value, but unsynchronized read and write operations can race. Threads in different processes have separate memory and need an explicit communication mechanism.
Account for the cost
- Each thread has a stack. A thread-per-request server can consume large amounts of memory during a spike.
- Creating a thread requires kernel bookkeeping and scheduler setup.
- Context switches consume CPU without advancing application work. A switch often takes about 1 to 10 microseconds, and many switches add up.
The naive one-thread-per-request model struggles when many requests wait at once. A spike of 10,000 requests can reserve 8 to 9 GB for stacks alone, depending on stack configuration. Threads remain useful for CPU-bound work because threads can execute in parallel across cores. For I/O-heavy work, an event loop or runtime-scheduled task can hold more in-flight operations at lower per-task cost.
Read the event loop and race conditions next.