TLDR Tech / Data
PositionService: Pagination at Scale
Arcesium re-architected their pagination service to handle billions of financial records and ~2 million API calls/day without breaking SLOs. The solution uses cursor-based pagination on date, fetches large indexed chunks (TOP ... WITH TIES), aggregates in SQL, writes the intermediate result to Parquet on S3, and serves later pages with DuckDB from S3. Stable commit timestamps and a uni-temporal model preserve point-in-time consistency across multi-page reads.
Read the full story