The canonical iterative-scaling example. Watch how a design evolves: benchmark → profile → fix → repeat. The AWS specifics are not required: the principles apply anywhere.
Define use cases and constraints
Use cases
- User makes a read or write request; the service processes, stores data, and returns results.
- The service must evolve from a few users to millions.
- Service has high availability.
Constraints and assumptions
- Traffic is not evenly distributed; relational data is needed.
- 10 million users, 1 billion writes/month, 100 billion reads/month → 100:1 read:write.
- 1 KB per write → 1 TB/month, 36 TB in 3 years.
- 400 writes/s, 40,000 reads/s.
Sketch the high-level design
- Client and routing
The client resolves DNS and can obtain configured static content from the CDN.
- Application
One web-server instance handles application traffic.
- Relational data
The instance reads and writes a SQL database.
- Content
Object storage holds stored content. A single application instance remains a capacity and availability limit.
The initial design keeps few deployed components while retaining distinct responsibilities.
How to read this model
A small service console beside relational and object archives. The object is a teaching analogy. Inspecting a part highlights its role; it does not execute or simulate the software. Cabinet trays are records, workpieces are data, and latches are checks. Unless an explanation states a quantity, compartments and sample workpieces do not denote data counts. Part dimensions are not a quantitative scale.
Define the core components (the journey)
Start with a single box
- One EC2 instance: a web server + a MySQL database.
- Vertical scaling: pick a bigger box; monitor CPU, memory, IO, network.
- Assign a public static IP (Elastic IP) and a DNS (Route 53).
- Secure the server: open only ports 80/443 (and 22 to whitelisted IPs); block outbound connections.
Users+: split the box
Benchmarks show MySQL eating memory/CPU and user content filling disk.
- Move static content to an Object Store (S3).
- Move MySQL to its own box (RDS) with multiple availability zones and encryption at rest.
- Use a VPC: public subnet for the web server, private subnet for everything else.
Users++: scale the web tier
The single web server bottlenecks at peak.
- Add a load balancer (ELB or HAProxy) with active-active/passive in multiple availability zones.
- Terminate SSL at the balancer.
- Add multiple web servers; separate web servers from application servers (web servers run as reverse proxies).
- Add master-slave failover for MySQL.
- Move static (and some dynamic) content to a CDN (CloudFront).
Users+++: scale the database
Read-heavy (100:1) → the database suffers on reads.
- Add a memory cache (ElastiCache) for hot content and sessions (web servers become stateless → autoscalable).
- Add MySQL read replicas to relieve the write master; split reads and writes in the app.
Users++++: autoscale
Traffic spikes during business hours.
- Add autoscaling: a group per web/app server type, min/max instances, trigger on CPU/latency/network (CloudWatch).
- Automate DevOps (Chef, Puppet, Ansible); monitor host, aggregate, and log levels.
- Edge and fleet
DNS, CDN, a load balancer, web servers, and autoscaling address delivery and web concurrency.
- Background work
A queue and worker service absorb appropriate slow or bursty work.
- Hot reads
Read/write APIs and memory cache reduce repeated database work.
- Data capacity
A SQL primary, replicas, NoSQL, and object storage support their respective data access patterns.
Each scaling stage should address an observed or justified constraint.
How to read this model
An expandable service rack beside a job magazine and replicated archive. The object is a teaching analogy. Inspecting a part highlights its role; it does not execute or simulate the software. Cabinet trays are records, workpieces are data, and latches are checks. Unless an explanation states a quantity, compartments and sample workpieces do not denote data counts. Part dimensions are not a quantitative scale.
Additional talking points
- SQL scaling patterns (sharding, federation) beyond read replicas: see database.
- Cache strategies: see cache.
- Security hardening at each step: see security.