Accelerator compute keeps pulling ahead of the memory that feeds it, and Micron put a ratio on the gap at Hot Chips 2026.
HBM design fellow Raghu Sreeramaneni told the Stanford conference that AI compute roughly triples every two years while HBM bandwidth grows about 2x, a mismatch that leaves processors waiting on data and real throughput far below peak.
The presentation quantified how much of an AI package is now memory. A base die with eight 12-high HBM4 stacks covers more than 12,000 square millimeters, making the DRAM the dominant component rather than the GPU.
Micron’s HBM4 runs at up to 2,800 GB/s per stack on a 2,048-bit interface, twice the channels of HBM3E, and volume shipments for Nvidia’s Vera Rubin platform are underway. The company’s 2026 HBM output is fully committed under contract.
Next come reliability and thermals. Micron uses on-die Reed-Solomon error correction plus system-level meta-bits per access, and cites fusion bonding for tighter interconnect pitch, with stacks beyond 16 dies under study.
Samsung and SK hynix also presented HBM sessions at the event, a rare convergence that highlights memory’s new role as the binding constraint on AI progress.
