Define use cases and constraints
Use cases (in scope)
- User enters a block of text and gets a randomly generated link.
- Default: never expires. Optional: timed expiration.
- User enters a paste's URL and views the contents.
- User is anonymous.
- Service tracks page analytics (monthly visit stats).
- Service deletes expired pastes.
- Service has high availability.
Out of scope
Account registration, email verification, login, editing, visibility settings, custom shortlinks.
Constraints and assumptions
- Traffic is not evenly distributed.
- Following a short link must be fast.
- Pastes are text only.
- Page-view analytics do not need to be real time.
- 10 million users, 10 million writes/month, 100 million reads/month → 10:1 read:write.
Usage
- ~1.27 KB per paste → 12.7 GB of new content per month, ~450 GB in 3 years.
- 360 million shortlinks in 3 years.
- 4 writes/second, 40 reads/second.
Sketch the high-level design
- Entry
The client reaches the web server.
- Operations
Write, read, and analytics APIs handle their separate request types.
- Metadata
SQL holds metadata for fast lookups.
- Bodies
Object storage holds paste content; the read path combines content with metadata.
SQL metadata locates a paste; object storage holds its body.
How to read this model
A read/write console beside a metadata cabinet and body archive. The object is a teaching analogy. Inspecting a part highlights its role; it does not execute or simulate the software. Cabinet trays are records, workpieces are data, and latches are checks. Unless an explanation states a quantity, compartments and sample workpieces do not denote data counts. Part dimensions are not a quantitative scale.
Define the core components
Write: user pastes text, gets a link
- Client → Web Server (reverse proxy) → Write API.
- Write API:
- Generates a unique URL and checks the database for a duplicate.
- Saves to the
pastestable and to an Object Store (S3). - Returns the URL.
shortlink char(7) NOT NULL
expiration_length_in_minutes int NOT NULL
created_at datetime NOT NULL
paste_path varchar(255) NOT NULL
PRIMARY KEY(shortlink)
URL generation
- Take the MD5 hash of
ip_address + timestamp(uniform, 128-bit). - Base62 encode it to
[a-zA-Z0-9]: safe for URLs, deterministic. - Take the first 7 characters → 62⁷ possible values, far more than the 360 million needed.
Read: user views a paste
- Client → Web Server → Read API.
- Read API checks the database for the URL; on a hit it fetches the contents from the Object Store, else returns an error.
Analytics
Realtime analytics are not required → MapReduce the web server logs to produce hit counts.
Expiration
Scan the table for entries whose expiration timestamp is past, then delete or mark them expired.
Scale the design
- Entry
DNS and CDN precede the load balancer and web servers.
- APIs
Write, read, and analytics APIs retain separate responsibilities.
- Hot reads and writes
Memory cache and SQL read replicas handle suitable reads; SQL writes go to the primary.
- Other storage
Object storage holds paste bodies; a warehouse supports analytics.
Caches and replicas protect the read path; analytics has its own storage.
How to read this model
An edge console beside cache, database, and body-storage holders. The object is a teaching analogy. Inspecting a part highlights its role; it does not execute or simulate the software. Cabinet trays are records, workpieces are data, and latches are checks. Unless an explanation states a quantity, compartments and sample workpieces do not denote data counts. Part dimensions are not a quantitative scale.
- Load balancer + multiple web servers; CDN; master-slave replicas.
- 40 reads/s (higher at peak) → serve popular pastes from a memory cache.
- 4 writes/s → a single master-slave is enough; else federation, sharding, denormalization, SQL tuning.
- Analytics → data warehouse (Redshift, BigQuery). Object store (S3) handles the 12.7 GB/month.
Additional talking points
- NoSQL alternatives: key-value or document store (discuss SQL vs NoSQL).
- Cache strategies and update policy: see cache.