# Parquet

> A file format that stores tables by column, compressed, so analytics reads far less.

- Level: Advanced
- Topics: Data
- URL: https://tidbit.legios.com.ar/parquet/
- In Spanish: [Parquet](https://tidbit.legios.com.ar/es/parquet/)

**Parquet** (Apache Parquet) is an open file format for tables, designed for analytics. A CSV writes a table row by row; Parquet writes it **column by column**: all the dates together, then all the amounts, then all the countries. It is binary rather than plain text, so you cannot open it in a text editor, but almost every data tool reads it: Spark, Amazon Athena, pandas, DuckDB and the major data warehouses.

Storing by column pays off twice. First, a query that needs two columns out of fifty reads only those two and skips the rest of the file. Second, values in a column look alike (the same few countries repeated millions of times), so they compress very well, often to a fraction of the CSV size. Each file also records the minimum and maximum of every column for each block of rows, so a query for March can skip blocks that only contain January.

Parquet also keeps the **schema** inside the file: column names and types, so a date stays a date and nobody has to guess. That is why it became the default format of the data lake: engines like Athena charge by data scanned, and converting raw logs from CSV or JSON to Parquet can make the same query both faster and much cheaper. The trade-off is that it is made for writing once and reading many times; changing a single row means rewriting a file.

## An analogy

A filing cabinet with one drawer per kind of information instead of one folder per person. To add up everyone's salary you open only the salary drawer, and since it holds nothing but numbers, it packs tightly.

## Keep in mind

- Columnar: a query reads only the columns it needs.
- Similar values sit together, so files compress far better than CSV.
- The schema travels inside the file, with real types.
- Built for write-once, read-many analytics, not for editing single rows.

## Go deeper

- [Parquet overview (Apache Parquet docs)](https://parquet.apache.org/docs/overview/)
- [Apache Parquet on Wikipedia](https://en.wikipedia.org/wiki/Apache_Parquet)

## Related terms

- [CSV](https://tidbit.legios.com.ar/csv/): A plain text file that holds a table: one row per line, commas between values.
- [Apache Spark](https://tidbit.legios.com.ar/apache-spark/): An open source engine that splits work on huge datasets across many machines.
- [Amazon Athena](https://tidbit.legios.com.ar/amazon-athena/): Run SQL directly on files sitting in S3, with no database to set up.
- [Data warehouse](https://tidbit.legios.com.ar/data-warehouse/): A database built for analysis: years of history, from many sources, in one place.
- [Database schema](https://tidbit.legios.com.ar/database-schema/): The blueprint of a database: which tables exist, their columns and how they connect.
- [Data lake](https://tidbit.legios.com.ar/data-lake/): Cheap storage where all your data is kept as raw files, to be sorted out later.
- [JSON](https://tidbit.legios.com.ar/json/): The plain-text format programs use to hand each other structured data.