Pattern
Medallion Architecture
Intent
Give a sprawling lakehouse a default shape: land data raw and immutable (bronze), standardise and deduplicate it (silver), and serve business-ready, modelled data (gold). Each layer has one job, and trust increases as data moves right.
Context
You’re building on a lakehouse — cheap object storage with an open table format — and many teams land, transform, and consume data in the same platform. Without a convention, pipelines read from arbitrary points and every incident becomes archaeology.
Structure
Bronze stores data exactly as ingested, append-only, so any downstream table can be rebuilt by replay — which is also what makes idempotent pipelines practical. Silver applies types, deduplication, and conformance. Gold holds dimensional models or aggregates that consumers actually query.
Trade-offs
Gains: provenance (raw history survives), a shared vocabulary across teams, clear blast-radius when a pipeline fails, and a natural place to enforce quality gates between layers.
Costs: three copies of much of your data; latency added at every hop; and a false sense of completion — the layers say nothing about modelling, ownership, or definitions, which is where warehouses actually fail. Teams routinely ship a perfect bronze/silver/gold pipeline that still delivers numbers nobody trusts.
When not to use it
Skip or collapse the layers when the source is already clean and structured (a CDC feed of a well-modelled OLTP system may not need a separate silver), when latency budgets are tight, or when the team is small enough that convention overhead outweighs archaeology risk. The layer count is a default, not a law — what matters is immutable raw history plus a declared, modelled serving layer.
Common questions
What are the bronze, silver, and gold layers?
Bronze holds raw data exactly as ingested, append-only and replayable. Silver holds cleaned, typed, deduplicated, conformed data. Gold holds modelled, business-ready tables — dimensional models or aggregates — that consumers actually query. Trust increases as data moves from bronze to gold.
Do I always need all three medallion layers?
No. The layer count is a default, not a law. A clean, well-modelled CDC source may not need a separate silver step, and latency-sensitive pipelines may collapse layers. What matters is keeping immutable raw history and a declared, modelled serving layer — however many stages sit between.
Is the medallion architecture the same as a lakehouse?
No — the lakehouse is the platform (open-format tables on object storage); the medallion architecture is a convention for organising data within it. You can run a lakehouse without medallion layers, and apply bronze/silver/gold thinking in a plain warehouse.
Essays by email
One new essay on data architecture, straight to your inbox. No noise, unsubscribe anytime.