As high-bandwidth memory stays scarce and costly, parts of the industry have floated a cheaper alternative for AI servers. Two engineers working on the problem say it cannot carry the load.
Speaking at the AI Infrastructure Summit in California, an OpenAI researcher working on accelerator design and Intel’s head of AI system-on-chip architecture argued that pooled memory will complement stacked memory rather than replace it.
The technology in question is Compute Express Link, an interconnect that attaches extra memory to a server and lets capacity be shared across systems. Accelerators reach it through the processor.
The OpenAI researcher said he has yet to find a compelling use for it in live model execution, suggesting its role may be limited to holding data that models rarely touch.
The Intel architect pointed to the bandwidth gap. Moving data between graphics processors over the link is far slower than reading stacked memory directly, which leaves the interconnect closer to a tier above storage.
Suppliers are treating it that way. SK hynix has described an expansion tier that inserts hybrid memory on the link between DDR and solid-state drives inside existing server stacks.
If that reading holds, leadership in stacked memory stays with the two Korean makers that dominate supply, and the new layer adds capacity without shrinking the stack at its top.
