TLDR Tech / Hardware
d-Matrix stacks logic on 3D DRAM to target 100 TB/s inference bandwidth
d-Matrix's Raptor accelerator stacks an N4 logic die directly on 3D DRAM and uses stream blocking, pinless data-bus inversion, thermal-aware refresh, and bank chaining to hold bandwidth near 100 TB/s. The company claims a 72-card rack can serve a 3-trillion-parameter-class model at 1M context at roughly 1,000 tokens per second per user, but the live Hot Chips report explicitly says working-system evidence and drawbacks remain to be shown.
Read the full story