TLDR Tech / Data
From Homegrown to Flink: Migrating a Stateful Ad Event Join at Scale
Zalando replaced a custom system built more than seven years ago for matching ad events with Apache Flink, while continuing to process up to 200 MB of data per second. The new design stores unmatched events until their counterparts arrive, uses disk-backed state for reliability, and checkpoints progress every three minutes. The migration cut daily EC2 costs by more than half and improved event matching by about 0.5%.
Read the full story