Server memory architecture is heading for a structural shift. The traditional hierarchy that puts HBM next to the GPU and DDR next to the CPU cannot scale to meet AI inference workloads that want terabytes of capacity per node. The emerging answer is a disaggregated memory pool connected through the Compute Express Link interconnect, and two of the world’s largest memory makers are racing to deliver it.
Samsung’s first CXL Memory Module-DRAM 3.0 chips will go into mass production before the year ends, using the CXL 3.2 interconnect protocol. The DIMM-sized module fits a standard DDR5 slot yet treats its DRAM as a network resource that any processor on the fabric can reach. Samsung says this architecture can raise a server’s usable memory ceiling by 50% and roughly double the bandwidth a CPU or GPU sees compared with a conventional DDR5 channel.
SK Hynix demonstrated second-generation CXL samples at the HPED 2026 conference last month and has begun customer discussions. The company also published details of an Inference Memory Tiering Expansion scheme that positions CXL memory as an intelligent intermediary between ultrafast HBM and high-latency SSD storage, squeezing 35.7% more inference efficiency out of existing GPU clusters.
The CXL 3.2 specification includes a hardware monitoring unit that tracks which data pages are accessed most frequently and moves them automatically between tiers. This logic reduces the amount of data shuttling between memory and storage, cutting both latency and power draw.
Nvidia baked CXL support into its upcoming Vera Rubin GPU platform, and both Intel and AMD have integrated CXL controller blocks into their most recent server CPU designs. Microsoft is running an internal pilot service that uses CXL memory pools to test whether splitting DRAM away from individual servers can lift overall fleet utilization in its cloud datacenters.
Interoperability across vendors remains the biggest obstacle. CPUs, switches, memory controllers, and operating system schedulers must all agree on CXL semantics before the technology becomes a standard server building block rather than a custom deployment.
