dataarchitect.studio

Field Notes

Parquet vs ORC: What the Benchmark Actually Showed

Both are columnar, both prune columns, both skip data with embedded statistics. The architecture is close enough that the choice usually gets settled by a piece of received wisdom instead: ORC compresses a bit better, Parquet has the better ecosystem.

The second half is true. The first half did not survive measurement. Benchmarked at matched compression codecs on a realistic 3-million-row table, Parquet was 23–26% smaller than ORC, and it also wrote faster and scanned faster. Not marginally, and not in the direction the guidance predicts.

For a new lakehouse: Parquet. Choose ORC when a Hive-centric stack expects it, not because you believe it compresses better.

Where the compression claim comes from

Most published comparisons compare Parquet with Snappy against ORC with Zlib. Zlib compresses harder than Snappy, so ORC’s file comes out smaller, and the difference gets written up as a format difference. It’s a codec difference.

Matched properly, at 1,000,000 rows:

Compression Parquet ORC Difference
Snappy 24.3 MB 32.9 MB Parquet 26% smaller
DEFLATE (gzip / zlib) 17.8 MB 23.1 MB Parquet 23% smaller

And at 3,000,000 rows across the full set of measurements:

Format Size Write Full scan 2 of 11 columns
ORC (snappy) 96.2 MB 1.84 s 0.676 s 0.109 s
Parquet (snappy) 72.3 MB 1.35 s 0.420 s 0.055 s
ORC (zlib) 68.1 MB 4.67 s 1.289 s 0.262 s
Parquet (zstd) 56.6 MB 1.32 s 0.377 s 0.054 s

The row worth staring at is ORC with zlib, because zlib is the default nobody changes: it writes 3.5× slower than Parquet with zstd and still produces a 20% larger file.

I want to be careful about the scope of this. It is one dataset shape, using pyarrow’s ORC writer at default settings — not Hive’s, and not tuned. ORC’s reputation was built in tuned Hive stacks on different data and may well hold there. What the measurement supports is narrower and still useful: “ORC compresses better” is not a default you can assume.

Parquet versus ORC at matched compression codecs Paired bars at one million rows. With Snappy, Parquet is 24.3 megabytes against ORC's 32.9, making Parquet 26 percent smaller. With DEFLATE, Parquet is 17.8 megabytes against ORC's 23.1, making Parquet 23 percent smaller. A note explains that most published comparisons instead pit Parquet with Snappy against ORC with Zlib, which measures a codec difference and reports it as a format difference. Matched codec, 1M rows — smaller is better Parquet · snappy 24.3 MB ORC · snappy 32.9 MB — Parquet 26% smaller Parquet · DEFLATE 17.8 MB ORC · DEFLATE 23.1 MB — Parquet 23% smaller How the usual comparison gets it backwards Parquet + Snappy vs ORC + Zlib → ORC looks smaller but that compares CODECS, not formats. Zlib compresses harder than Snappy. Match the codec and the result reverses. One dataset, pyarrow's ORC writer, defaults.
The comparison everyone runs measures the codec. Match it, and the ranking flips.

What genuinely separates them

Strip out the compression argument and three real differences remain.

Ecosystem gravity. Parquet is the default across Spark, the cloud warehouses, dbt, and every open table format. ORC’s home is Hive and Trino. This is the difference that actually decides most cases, and it always was.

Hive ACID. ORC supports ACID transactions within Hive, which mattered a great deal before table formats existed. It matters much less now that Iceberg and Delta provide transactions over any underlying file format — including ORC, but conventionally Parquet.

Tooling depth. More readers, more writers, more debugging tools, more people who have hit your problem before. Unglamorous and worth a lot at 2am.

The codec choice matters more than the format choice

This is the part of the measurement with the largest practical payoff, and it applies to both formats:

Codec (Parquet, 1M rows) Size Write time
zstd 19.3 MB 0.54 s
gzip 17.8 MB 16.00 s

Gzip bought an 8% smaller file for a 30× slower write. For analytical tables, written once and read many times, that is a bad trade; for continuously written tables it is an indefensible one.

-- Usually a bigger win than switching format:
ALTER TABLE lake.sales.orders
  SET TBLPROPERTIES ('write.parquet.compression-codec' = 'zstd');

If you’re on Parquet with gzip or ORC with zlib because that was the default when the pipeline was written, changing that one property is probably the cheapest performance work available to you, and nothing downstream has to change.

The decision rule

Greenfield lakehouse, or any Spark / cloud-warehouse / dbt stack?
   → Parquet, with zstd. Do not deliberate.

Established Hive or Trino platform that already standardised on ORC,
or you depend on Hive ACID?
   → Stay on ORC. Migrating for a compression claim that
     did not survive measurement is not worth the churn.

Choosing ORC because you read that it compresses better?
   → Measure it on your data first. On mine it was 23-26% larger.

The broader point, which is why this essay exists at all: the ORC compression claim is repeated in nearly every comparison of the two formats, including one I published on this site before I measured anything. It propagated because it sounds plausible and because the benchmark that would test it takes an afternoon that nobody spends. The script is published and the dataset is seeded. If your table behaves differently, that is worth knowing, and I’d rather be corrected than repeated.

Common questions

Is Parquet or ORC better?

For a new lakehouse, Parquet, and by a wider margin than the usual advice suggests. Benchmarked at matched compression codecs on a realistic 3-million-row table, Parquet was 23-26% smaller than ORC, wrote faster, and scanned faster. ORC remains the right choice where a Hive-centric stack already expects it, or where you need Hive's ACID support, but the common claim that ORC compresses better did not survive measurement.

Does ORC compress better than Parquet?

Not on the data measured here, and the claim is repeated far more often than it is tested. Most published comparisons pit Parquet with Snappy against ORC with Zlib, which compares codecs rather than formats. Matched properly: Snappy to Snappy, Parquet was 26% smaller; DEFLATE to DEFLATE, 23% smaller. This was one dataset with pyarrow's ORC writer at defaults, so measure your own data before it drives a decision.

When should I still use ORC?

When the surrounding stack expects it. ORC has deep roots in Hive and Trino deployments and supports ACID transactions within Hive, so an established Hive-centric platform is a legitimate reason to stay. What is no longer a good reason is an inherited belief about compression ratio.

Are Parquet and ORC both columnar?

Yes. Both store values column by column rather than row by row, both prune columns at read time, and both skip data using embedded statistics. That shared design is why the practical differences come down to ecosystem, tooling maturity, and measured behaviour on your data rather than to architecture.