TLDR Tech / AI
Optimizing Models to Be Fast at Codegen
Morph LLM enhances open coding models by training a drafter on coding output, not general internet data, to achieve a 3.07x speedup in speculative decoding. Autoresearch automates kernel tuning for low-demand GPUs, boosting warp-decode kernels to 162 tok/s on affordable hardware. Interconnect over PCIe replaces expensive NVLink with custom kernels, maintaining performance by sharing caches via TCP and cutting time-to-first-token by 84%.
Read the full story