TLDR Tech / AI
Taalas HC1: Absurdly Fast, Per-User Inference at 17,000 tokens/second
Taalas has been quietly building a model-on-silicon chip that bakes an LLM directly into hardware to deliver absurdly fast, per-user inference. The HC1 chip runs Meta's Llama 3.1 8B at extreme speed by effectively hardwiring the model into silicon. It delivers inference at around 17,000 tokens per second per user. This is just the first version - the next iterations are already geared toward much better fidelity.
Read the full story