TLDR Tech / Practical
Pulsar engine boosts Mac LLM inference speed with new techniques
The article details the performance enhancements in Pulsar, an inference engine designed for efficient large language model execution on Mac devices. Through methods such as specialized Q4 projections, speculative decoding, and deferred Gated DeltaNet state commits, Pulsar significantly outperformed runtimes like MLX-LM, llama.cpp, and Lithos. The analysis covers micro-architectural trade-offs, measurement complexities, and results that vary by workload.
Read the full story