TLDR Tech / Data
Apache DataFusion 54.0.0 Released
Apache DataFusion 54.0.0 adds major SQL upgrades, including LATERAL joins, SQL lambda functions for arrays, a new Arrow-based Avro reader, and spill-to-disk for memory-heavy nested loop joins. Performance also jumps, with near-unique LEFT/FULL sort-merge joins up to 20–50× faster and repartition-heavy operations improving by up to 50%.
Read the full story