TLDR Tech / Data
Apache Airflow 3.2.0: Data-Aware Workflows at Scale
Apache Airflow 3.2.0 adds asset partitioning for data-aware scheduling, so downstream DAGs trigger only for the exact partition that changed rather than every upstream update. It also introduces experimental multi-team support for isolating DAGs, connections, variables, pools, and executors in a single deployment, plus synchronous deadline alert callbacks via the executor. Rendered task instance field cleanup is now about 42x faster for heavily mapped DAGs, and PythonOperator now supports async callables.
Read the full story