Compaction (Data Lake / Table Format)

Compaction is an optimization process used within open table formats like Apache Iceberg on data lakes. It involves merging many small data files into fewer larger files, which improves query performance by reducing metadata overhead and optimizing read operations. This is typically a background maintenance task essential for maintaining efficiency in large-scale lakehouse tables.