Define use cases and constraints
Use cases (in scope)
- Service calculates the past week's most popular products by category.
- User views that list.
- Service has high availability.
Out of scope
The general e-commerce site: design only the sales-rank feature.
Constraints and assumptions
- Traffic is not evenly distributed.
- Items can be in multiple categories; items cannot change categories; no subcategories.
- Results update hourly (popular products may update more often).
- 10 million products, 1,000 categories, 1 billion transactions/month, 100 billion reads/month → 100:1 read:write.
Usage
- ~40 bytes per transaction → 40 GB/month, 1.44 TB in 3 years.
- 400 transactions/s, 40,000 reads/s.
Sketch the high-level design
- Entry
The client reaches the web server, Sales API, and Read API.
- Sales records
Persist structured sales data in SQL and raw logs in object storage.
- Aggregation
The sales-rank service processes raw logs in batch.
- Ranking reads
Read the published category rankings through the read path.
The example publishes hourly category rankings from raw sales logs.
How to read this model
A sales-input feeder beside an aggregation fixture and rank archive. The object is a teaching analogy. Inspecting a part highlights its role; it does not execute or simulate the software. Cabinet trays are records, workpieces are data, and latches are checks. Unless an explanation states a quantity, compartments and sample workpieces do not denote data counts. Part dimensions are not a quantitative scale.
Define the core components
Compute the rank
Store raw Sales API logs in an Object Store (S3), then run a two-step MapReduce:
- Transform to
(category_id, product_id) → sum(quantity). - Do a distributed sort to rank products per category.
Write the result into the sales_rank table (id, category_id, total_sold, product_id), indexed on id, category_id, product_id.
View the rank
Read API reads the sales_rank table by category_id.
Scale the design
- Entry
DNS, CDN, a load balancer, and web servers precede the APIs.
- Write path
The Sales API writes through the SQL primary and raw-log storage path.
- Batch path
Sales-rank jobs aggregate object-storage logs into an hourly rank table.
- Read path
The Read API uses memory cache and appropriate SQL replicas to serve published results.
Batch aggregation and interactive ranking reads have different capacity needs.
How to read this model
An edge console beside a batch fixture and cached rank drawer. The object is a teaching analogy. Inspecting a part highlights its role; it does not execute or simulate the software. Cabinet trays are records, workpieces are data, and latches are checks. Unless an explanation states a quantity, compartments and sample workpieces do not denote data counts. Part dimensions are not a quantitative scale.
- 40,000 reads/s → serve popular ranks from a memory cache; read replicas handle misses.
- 400 writes/s → heavy for a single write master → federation, sharding, denormalization, SQL tuning, and consider NoSQL.
- Keep a limited time window in the database; archive the rest to a data warehouse (Redshift/BigQuery) or the object store.