TLDR Tech / Data
Optimizing for Almost Sorted Data: Sort Pushdown in Apache DataFusion
Apache DataFusion now exploits exact and partial sortedness automatically for ORDER BY and ORDER BY ... LIMIT N, using Parquet min/max statistics, file reordering, and dynamic filters to avoid full scans and blocking sorts.
Read the full story