TLDR Tech / AI
The Inference Shift
The most important aspect for agentic inference is memory. Agents need context, state, and history, which requires a memory hierarchy wrapped around a model. This implies a necessary trade-off of speed for capacity, but agentic tasks don't really need to be that fast if a human isn't involved.
Read the full story