TLDR Tech / DevOps
BitNet
Microsoft released bitnet.cpp, an official inference framework for 1-bit LLMs like BitNet b1.58, achieving speedups of 1.37x to 6.17x on ARM and x86 CPUs while reducing energy consumption by up to 82.2%. The framework can run a 100B parameter model on a single CPU at human reading speed (5-7 tokens per second), with the first release focusing on CPU support and GPU/NPU compatibility coming later.
Read the full story