TLDR Tech / Data
Inside the flight path of real-time ingestion in Apache Pinot
To guarantee a single consistent "winning" segment across multiple Pinot server replicas consuming the same Kafka partition, Pinot uses a lightweight controller-orchestrated blocking commit protocol where the controller elects a committer based on the max offset, while non-committers discard locals and download the official version.
Read the full story