TLDR Tech / AI
FreeToken: Efficient Edge-Native MoE Serving
FreeToken continuously remaps experts, model state, CPU/GPU work, and agent state reuse to the bandwidth and memory actually available on a personal machine. The authors report support for more than 20 MoE models, from 35B models on an 8GB laptop GPU to a 753B GLM model on one workstation GPU.
Read the full story