A representative disaggregated object-storage architecture
Large object stores commonly separate metadata lookup from payload placement. The topology below is a representative design, not a claim that AWS publicly uses these exact chunk sizes, coding parameters, databases, or placement rules for every S3 object.
S3 Disaggregated System Topology
flowchart TD
Client["Client HTTP Request: PUT /photos/vacation.jpg"] --> Edge["S3 Frontend API Gateway"]
subgraph MetadataPlane ["Metadata Plane (Index Subsystem)"]
Edge -->|"1. Resolve bucket & permissions"| MetaEngine["LSM-Tree Metadata Index (Sharded Key-Value)"]
MetaEngine -->|"2. Record object metadata, version, ETag"| MetaStore[("Distributed Inode & Parts DB")]
end
subgraph PayloadPlane ["Payload Storage Plane (Chunk Nodes)"]
Edge -->|"3. Stream 64MB Chunks"| ChunkManager["Storage Node Orchestrator"]
ChunkManager -->|"4. Reed-Solomon Erasure Coding (8+4)"| StorageNodes["Fragments placed across independent failure domains"]
end
Internal Subsystems of S3
- Frontend API Routers: Thousands of stateless reverse proxies terminating TLS, validating AWS SigV4 cryptographic signatures, and enforcing per-prefix rate limits.
- Metadata Subsystem (Index): A specialized key-value store optimized for lexicographical prefix scanning. Because S3 supports list operations (
GET /bucket?prefix=users/), the metadata tier organizes keys in partitioned B-trees or LSM-trees rather than simple hash rings. - Storage Node Fleet: Commodity servers packed with 3.5-inch high-density HDDs. Each node runs ShardStore, a Rust key-value store that maps shard identifiers to data using an LSM-tree over extents it manages directly on disk.
Worked example: the life of a PUT
- DNS routes
PUT /my-bucket/logs/2026/09/30/app.gzto a front-end fleet host, which terminates TLS and verifies the SigV4 signature and bucket policy. - The front end asks placement for storage targets and streams the body, splitting it into shards that are replicated or erasure coded across devices in different Availability Zones (for S3 Standard, at least three AZs).
- Only after enough shards are durably stored does the front end write the index entry mapping the key and version to the shard locations.
- The index write is the commit point; the client receives
200 OKwith an ETag. Since 2020, a subsequent GET or LIST observes the new object (strong read-after-write consistency).
Public numbers worth knowing
| Fact | Value |
|---|---|
| Maximum object size | 5 TB |
| Maximum single PUT | 5 GB; larger objects use multipart upload |
| Multipart parts | Up to 10,000 parts, 5 MB to 5 GB each (last part may be smaller) |
| Request rate per partitioned prefix | At least 3,500 PUT/COPY/POST/DELETE and 5,500 GET/HEAD per second |
| Designed durability (S3 Standard) | 11 nines annually |
| Scale cited by AWS in recent talks | Hundreds of trillions of objects, over 100 million requests per second at peak |
What ShardStore actually is
ShardStore is the storage-node software that S3 runs on its disks: a Rust key-value store, organized as an LSM-tree over disk extents, that maps shard identifiers to their data. AWS described validating it with lightweight formal methods in an SOSP 2021 paper. It is not the object metadata index.
Failure modes the architecture is designed for
- Drive and host loss: constant; repaired by rebuilding shards elsewhere from surviving shards.
- AZ loss: shards spread across AZs so objects remain readable.
- Hot prefixes: the index automatically partitions busy key ranges; sudden spikes can see 503 Slow Down responses until it adapts, so clients retry with backoff.
- Heat skew on drives: Warfield describes spreading new data so that each drive holds a mix of hot and cold data, balancing I/O across a fleet of millions of disks.