TLDR Tech / AI
Optimizing On-Device Inference for Apple Silicon
Apple's Lily engine optimizes on-device LLM inference for Apple silicon. It speeds up processing by leveraging Apple silicon's unified memory and specialized hardware, outperforming MLX-LM in prefill and decode throughput. Qwen3.6-35B-A3B model's unique architecture, including sparse MoE routing and Gated DeltaNet layers, enables advanced engine tuning for efficient model execution on one Mac.
Read the full story