What does an Iceberg catalog actually do?

It holds the authoritative pointer to each table's current metadata file and swaps that pointer atomically on every commit — which is what makes transactions work. On top of that mechanical job, catalogs layer the governance: who may read, write, or commit to which table, from which engine, with what audit trail.

Is an Iceberg catalog the same as a data catalog like Atlan or Collibra?

No — unfortunate naming collision. An Iceberg catalog is technical infrastructure: it's in the write path of every transaction. A data catalog is a discovery and documentation layer for humans. You'll likely have both, and the data catalog will read from the Iceberg catalog.

Can I switch Iceberg catalogs later?

Yes, but it's the painful migration in a lakehouse. The data files don't move, but every engine's connection config, all access-control policies, and any audit or lineage history live in the catalog. That switching cost — not the table format — is where lock-in lives in 2026, so choose deliberately.

Which Iceberg catalog should I default to?

Match your platform gravity: Unity if you're Databricks-centred, Glue if you're deep in AWS, Polaris (or Snowflake's managed Open Catalog) for a neutral, multi-engine REST catalog, Nessie if you specifically want git-like branching of your whole catalog. Neutral-by-default: Polaris.

How to Choose an Iceberg Catalog: Unity vs Polaris vs Glue vs Nessie

The table format decision got easy in 2026 — which means the hard decision moved. Every Iceberg table has exactly one authoritative record of “which metadata file is current,” and the component that holds it — the catalog — is now where governance, auditability, and the real lock-in of a lakehouse live. The main contenders: Unity Catalog (Databricks), Apache Polaris (open-sourced by Snowflake), AWS Glue, and Nessie. The decision rule, up front: follow your platform gravity — Unity for Databricks estates, Glue for deep-AWS shops — and when you want genuine engine neutrality, default to Polaris. The rest of this essay is why, and what each choice costs.

What a catalog actually is

Strip away the marketing and an Iceberg catalog does one mechanical thing: for every table, it stores a pointer to the current metadata file, and it swaps that pointer atomically when a commit happens. That single atomic swap is what gives an open table format ACID guarantees on dumb object storage. Everything else a catalog offers — access control, credential vending, auditing, lineage — is governance layered onto its position in the write path.

That position is why the choice matters: every engine, every pipeline, every commit goes through it. (Don’t confuse it with a data catalog like Atlan or Collibra — same word, different layer: one is transaction infrastructure, the other is documentation for humans.)

The comparison, honestly

	Unity Catalog	Apache Polaris	AWS Glue	Nessie
Steward	Databricks (OSS core)	Apache Software Foundation	AWS	Dremio-backed OSS
Formats	Delta, Iceberg, Hudi	Iceberg only	Iceberg + others	Iceberg
Iceberg REST spec	Yes, endpoint	Yes — reference implementation	Yes	Yes
Access control	Strong, platform-integrated	Granular, engine-neutral	IAM-based	Basic
Managed offering	Databricks	Snowflake Open Catalog	AWS	Dremio
Distinctive strength	Lineage + governance in one platform	Neutrality	Zero-effort if you’re on AWS	Git-like branches and tags
Gravity	Databricks	None — that’s the point	AWS	Dremio / advanced teams

The structural fact underneath the table: the Iceberg REST catalog specification is what every engine now standardises against, so any spec-faithful catalog can serve Spark, Trino, Flink, and friends. The differences are in governance depth, ecosystem gravity, and who you’re comfortable depending on.

What connecting looks like

The catalog is one config block per engine — which is also the honest preview of a future migration:

-- Spark: attach a REST catalog (Polaris, Unity's endpoint, Glue — same shape)
SET spark.sql.catalog.lake = org.apache.iceberg.spark.SparkCatalog;
SET spark.sql.catalog.lake.type = rest;
SET spark.sql.catalog.lake.uri = https://catalog.example.com/api/catalog;
SET spark.sql.catalog.lake.credential = <client-id>:<client-secret>;

-- After that, tables behave identically regardless of catalog vendor:
CREATE TABLE lake.sales.orders (
  order_id BIGINT, order_ts TIMESTAMP, amount DECIMAL(12,2)
) PARTITIONED BY (days(order_ts));

Identical SQL, whichever catalog answers that URI. The switching pain is never the tables — it’s re-plumbing every engine’s config and rebuilding every access policy and audit trail that accumulated in the old catalog.

The catalog decision follows gravity — with neutrality as the deliberate exception.

The honest trade-offs

Unity buys the deepest integrated governance — lineage, policies, audit in one place — priced in Databricks gravity. Glue is the lowest-effort answer inside AWS and the least interesting outside it. Polaris is the neutral choice: ASF-governed, REST-native, granular permissions, no engine agenda — but younger, and you’ll assemble more of the surrounding tooling yourself. Nessie gives you catalog-level branches and tags (test a whole pipeline against a branch of production data, then merge) — a genuinely distinctive capability that most teams don’t need yet.

Whichever you pick, pick it deliberately. In a lakehouse built on open formats, the catalog is the one component you can’t casually swap — it’s exactly where the next five years of governance decisions will accumulate.

What a catalog actually is

The comparison, honestly

What connecting looks like

The honest trade-offs

Common questions