TLDR Tech / Hardware
RPI and IBM propose REACH, a memory controller that cuts HBM error-correction overhead for LLM inference
Stronger HBM error correction usually means long-span codes that protect more efficiently, but coupling every small memory access to span-wide state and decoding at full HBM bandwidth is normally too costly to implement directly. REACH sidesteps that by exploiting a property specific to LLM decode workloads: reads are sequential, so they can be aggregated across a span cheaply, while writes are sparse, keeping parity-update traffic low. The controller uses standard inner codes to fix common errors and flag whichever chunks it can't resolve, then falls back to a long outer code reserved specifically for repairing those flagged erasures, avoiding the cost of running heavy decoding on every access.
Read the full story