TLDR Tech / Data
Volga - Data Processing for Real-Time AI/ML
Volga is now a fully Rust-based engine for real-time AI/ML, replacing the earlier Python+Ray core to get a simpler, higher-performance runtime and tighter control over execution and state. It unifies streaming, batch, and request-time compute in one standalone system, aiming to eliminate the usual stitching across Flink, Spark, Redis, and custom services while keeping point-in-time-correct state inside the engine. The key building blocks are Apache DataFusion for SQL pipelines, Apache Arrow for execution semantics, and SlateDB for S3-backed state.
Read the full story