SYSTEM DESIGN · QUESTION 117
How would you design a scalable object-storage service?
On this page
I would clarify object size, consistency, durability, access patterns, and lifecycle requirements. A control plane authenticates requests and manages metadata; a data plane streams objects to replicated or erasure-coded storage nodes. Use checksums, versioning, placement and repair processes, and lifecycle policies. Metadata consistency and recovery matter as much as storing bytes, especially during node and region failures.
LET'S MAKE IT SIMPLE
First, the idea in plain English
Object storage keeps files as named objects rather than rows in a database. The system must find each object, store its bytes durably, and return it efficiently. Metadata and content have different access patterns, so they may use separate services.
A USEFUL COMPARISON
Picture it this way
A warehouse has a catalog that records where each package is stored and a physical system that holds the packages. If one shelf fails, the warehouse needs another copy or a way to rebuild it from redundant pieces.
FOLLOW THE JOURNEY
Separate metadata control from durable object-byte storage
Trace object writes, placement, integrity checks, reads, and repair.
This simplified visual highlights the main relationships. In practice, the exact design depends on the requirements; Durability requires continuous health management.
A control plane authorizes and records metadata while a data plane writes checksummed object chunks across storage nodes; reads verify data and repair restores redundancy.
How to read this visual
This simplified visual highlights the main relationships. In practice, the exact design depends on the requirements; Durability requires continuous health management.
UNDER THE HOOD
How it works, step by step
Authorize and allocate
Validate identity, namespace, quota, and object key.
Why it matters: Control plane policy governs who can store and retrieve.
Write durable data
Stream bytes to replicated or erasure-coded placements with checksums.
Why it matters: Redundancy and integrity checks protect against failures.
Commit metadata
Publish object version and location after storage requirements pass.
Why it matters: Readers should not see a partially written object.
Read and repair
Fetch, verify, and reconstruct or repair degraded copies.
Why it matters: Durability requires continuous health management.
OUTSIDE THE TEXTBOOK
A practical example
A client uploads a large archive. The control plane checks access and selects a storage namespace; the data plane streams chunks, computes checksums, and places redundancy across failure domains. Once policy is satisfied, metadata exposes the object version. Background repair restores redundancy after a node failure.
GO A LITTLE DEEPER
Technical details that matter
- Replication uses full copies; erasure coding trades computation and reconstruction complexity for storage efficiency.
- Object keys and versioning policies influence overwrite, consistency, and deletion semantics.
- Durability claims require placement across independent failure domains and tested repair capacity.
MAKE A GOOD CHOICE
When is this useful?
- Backups, media, documents, and data-lake files.
- Large immutable blobs with high durability requirements.
- Content delivery origins and archival storage.
A filesystem suits workloads needing directory and in-place update semantics; relational storage suits small transactional objects.
WATCH OUT FOR
Common misunderstandings
- Confusing object storage with a POSIX filesystem.
- Acknowledging writes before durability or metadata commit requirements are met.
- Ignoring checksum verification and repair capacity.
REAL-WORLD TRADE-OFFS
Limitations and failure cases
- Object operations may not support arbitrary in-place updates.
- Metadata lookup and listing can become scaling bottlenecks.
- Cross-region durability increases cost and consistency complexity.
WHAT TO REMEMBER
Key takeaways
- 1Separate control-plane metadata from data-plane bytes.
- 2Use checksums and redundancy across failure domains.
- 3Plan versioning, deletion, repair, and consistency explicitly.
PRACTICE THE FOLLOW-UP
What might an interviewer ask next?
Replication versus erasure coding?
Replication stores full copies and simplifies reads and repair; erasure coding can reduce storage overhead but requires more computation and reconstruction.
Why separate metadata from object data?
Metadata needs low-latency lookup and transactional coordination, while object bytes need high-throughput streaming and durable placement.
SAY IT OUT LOUD
How to explain this in an interview
“I would separate a metadata control plane from a data plane that streams object bytes. Writes are authorized, checksummed, and placed with redundancy across failure domains before metadata makes the version visible. Reads verify integrity, and background repair restores redundancy. I would clarify consistency, versioning, listing, lifecycle, and recovery requirements.”
Connect this question to the roadmap topic it supports.