How does object storage achieve eleven nines of durability?

Not with a single trick but with a chain: an object is chunked and erasure-coded into k data + m parity shards, the shards are placed across independent failure domains (racks, then Availability Zones), and a background repair process rebuilds any lost shard fast — before enough others fail to cross the threshold. Amazon S3 is "designed to provide 99.999999999% durability" by combining these across a minimum of three Availability Zones.

Why not just keep three copies of every object (3x replication)?

At exabyte scale the cost is prohibitive. Three full copies is 200% storage overhead; erasure coding like Reed-Solomon RS(6,3) gives comparable fault tolerance at no more than 50% overhead — HDFS notes a 6-block file costs 18 blocks replicated but only 9 blocks erasure-coded. Replication is still used for tiny objects, where per-shard overhead and extra read I/O make coding a net loss.

What is the metadata/data split in object storage?

Two planes that scale differently. The data plane holds the object bytes (huge, sequential, erasure-coded). The metadata plane is the index that maps bucket+key+version to the shard locations — small per object but the hot path for every request and the true durability bottleneck: if the metadata is lost the healthy bytes become unfindable, so the metadata itself must be replicated and strongly consistent.

How does erasure coding survive losing a whole data centre?

By placement. With RS(6,3) an object becomes 9 shards; spread three-per-Availability-Zone across three AZs, losing an entire AZ removes exactly 3 shards, leaving 6 — precisely the k needed to reconstruct the object. Any k of the n shards rebuilds the original, so the AZ loss is a rebuild, not a data-loss event. S3 is "designed to sustain data in the event of the loss of an entire Amazon S3 Availability Zone."

Why is repair speed the real lever for durability, not the number of copies?

You only lose data if more than m shards fail inside the repair window — the time to rebuild the first failure. Backblaze models this directly: with a ~0.41% effective annual drive-failure rate and a 6.5-day average rebuild, four drives must fail before the first is rebuilt to lose a file, which works out to eleven nines. Halve the repair window and durability climbs by orders of magnitude with no extra storage.

Design S3-Like Object Storage

From one server writing files on disk to an exabyte store that survives an entire data-centre dying — the metadata/data split, erasure coding, placement across failure domains, and repair as the real durability lever.

System design · Systems. The source ↗

A free, interactive, animated visual explainer of Design S3-Like Object Storage — built to be understood, not skimmed.

Questions

How does object storage achieve eleven nines of durability?: Not with a single trick but with a chain: an object is chunked and erasure-coded into k data + m parity shards, the shards are placed across independent failure domains (racks, then Availability Zones), and a background repair process rebuilds any lost shard fast — before enough others fail to cross the threshold. Amazon S3 is "designed to provide 99.999999999% durability" by combining these across a minimum of three Availability Zones.
Why not just keep three copies of every object (3x replication)?: At exabyte scale the cost is prohibitive. Three full copies is 200% storage overhead; erasure coding like Reed-Solomon RS(6,3) gives comparable fault tolerance at no more than 50% overhead — HDFS notes a 6-block file costs 18 blocks replicated but only 9 blocks erasure-coded. Replication is still used for tiny objects, where per-shard overhead and extra read I/O make coding a net loss.
What is the metadata/data split in object storage?: Two planes that scale differently. The data plane holds the object bytes (huge, sequential, erasure-coded). The metadata plane is the index that maps bucket+key+version to the shard locations — small per object but the hot path for every request and the true durability bottleneck: if the metadata is lost the healthy bytes become unfindable, so the metadata itself must be replicated and strongly consistent.
How does erasure coding survive losing a whole data centre?: By placement. With RS(6,3) an object becomes 9 shards; spread three-per-Availability-Zone across three AZs, losing an entire AZ removes exactly 3 shards, leaving 6 — precisely the k needed to reconstruct the object. Any k of the n shards rebuilds the original, so the AZ loss is a rebuild, not a data-loss event. S3 is "designed to sustain data in the event of the loss of an entire Amazon S3 Availability Zone."
Why is repair speed the real lever for durability, not the number of copies?: You only lose data if more than m shards fail inside the repair window — the time to rebuild the first failure. Backblaze models this directly: with a ~0.41% effective annual drive-failure rate and a 6.5-day average rebuild, four drives must fail before the first is rebuilt to lose a file, which works out to eleven nines. Halve the repair window and durability climbs by orders of magnitude with no extra storage.

Design S3-Like Object Storage

Questions

Related explainers