▣ dataarchitect.studio

Pattern

Lakehouse

Keep one copy of data as open-format tables on cheap object storage, and let every engine — SQL, Spark, ML — read and write it with warehouse-grade guarantees.

Intent

End the two-copy world — one lake for ML, one warehouse for BI — by storing data once, in an open table format on object storage, with ACID transactions and schema enforcement, queryable by any engine.

Context

You run both structured analytics and lake-style workloads (ML, data science, semi-structured data), and keeping copies in sync between a lake and a warehouse has become its own engineering programme. Vendor lock-in on the storage layer is a real concern.

Structure

SQL / BI Spark / ML Streaming open table format + catalog ACID · schema · time travel · governance object storage — one copy, Parquet files every engine reads and writes the same tables; no replication between systems

The load-bearing component is the table format (Iceberg, Delta) plus the catalog that holds each table’s atomic pointer and enforces who may commit — which is where governance, and the remaining lock-in risk, now live.

Trade-offs

Gains: one copy of data; engine choice per workload; cheap storage economics; open formats outlive any vendor; time travel and safe schema evolution by default.

Costs: more moving parts than a turnkey warehouse (format, catalog, engines, maintenance jobs like compaction); performance tuning is your job; and the convenience gap with a mature warehouse, while narrowing, is real. Organising what goes where becomes its own discipline — usually the medallion pattern.

When not to use it

Purely structured BI at moderate scale with one engine: a plain warehouse is simpler, faster to run, and easier to staff. Adopt the lakehouse when the second copy or the second engine actually appears — not before.

Common questions

What is a data lakehouse in simple terms?

One copy of data, stored as open-format tables (Iceberg, Delta) on cheap object storage, with warehouse-grade guarantees — ACID transactions, schema enforcement, time travel — queryable by any engine: SQL for BI, Spark for ML, streaming for pipelines.

Do I need a lakehouse if I already have a data warehouse?

Only when the second workload actually appears. If all your work is structured BI in one engine, a warehouse is simpler and easier to run. The lakehouse earns its moving parts when ML, data science, or multi-engine access would otherwise force a second copy of the data.

Which table format should a lakehouse use?

Apache Iceberg has become the neutral standard — Databricks, Snowflake, BigQuery, and Fabric all read and write it. Delta Lake remains excellent inside Databricks, with UniForm bridging the two. The more consequential choice now is the catalog that governs the tables.