TLDR Tech / Practical
Colibri: Engine for Frontier MoE Models on Consumer PCs
Colibri is a C-based engine with no external dependencies, built to run advanced mixture-of-experts models on ordinary hardware. It achieves this by storing dense layers in RAM and streaming routed experts from disk storage. The engine supports a variety of language, image, and decision-making models across CPU, CUDA, Vulkan, and Metal platforms. It offers APIs compatible with OpenAI and Anthropic, along with local web interfaces and endpoints for system-one decision scoring.
Read the full story