Skip to main content

One post tagged with "S3"

Amazon S3 object storage

View All Tags

S3 Is Not an Object Store. It's a Consensus Store.

· 14 min read
Arun Lakshman Ravichandran
Software Engineer, AWS

Distributed coordination requires locks, leader election, and consistent configuration. The standard approach: deploy etcd, ZooKeeper, or provision a DynamoDB table with conditional expressions. Teams treat S3 as file storage.

Object stores now support consensus primitives.

Google Cloud Storage (GCS) has supported conditional writes via generation-match preconditions since its early releases. Azure Blob Storage has supported If-Match / If-None-Match on ETags for the same duration. S3 was the last to adopt. In May 2024, materializedview.io identified the gap: "S3 has no compare-and-swap operation, something every single other competitor has."

S3 closed the gap in two releases. If-None-Match shipped in August 2024. Full If-Match Compare-And-Swap (CAS) shipped in November 2024. AWS described the feature as "reliably offloading compare and swap operations to S3."

All three major object stores now support CAS. Herlihy proved in 1991 that CAS is a universal primitive: the single operation sufficient to build any concurrent data structure, wait-free.

Every major object store provides a universal coordination primitive. Most teams have not recognized this.