TLDR Tech / Data
Backfilling: the most underrated feature of a data pipeline
Backfills should use the same production pipeline as incremental loads, with configuration controlling whether it processes new data, everything, or a single record. This avoids fragile one-off scripts and, with bi-temporal writes, makes reprocessing safe and repeatable.
Read the full story