KAIST startup Panmnesia (the name means “the power to remember absolutely everything you think, feel, encounter and experience”) claims to have developed a new approach to boosting GPU memory.
The company's advancement enables terabyte-scale memory additions using cost-effective storage media such as NAND-based SSDs while maintaining reasonable performance levels.
However, there is a problem: the technology is based on the relatively new Compute Express Link (CXL) standard, which has not yet been proven in widespread applications and requires specialized hardware integration.
Technical challenges remain
CXL is an open standard interconnect designed to efficiently connect CPUs, GPUs, memory, and other accelerators. It allows these components to share memory coherently, meaning they can access shared memory without needing to copy or move data, reducing latency and increasing throughput.
Because CXL is not a synchronous protocol like JEDEC’s DDR standard, it can support multiple types of storage media without requiring exact time synchronization or latency. Panmnesia claims that initial testing has shown that its CXL-GPU solution can outperform traditional GPU memory expansion methods by more than three times.
For its prototype, Panmnesia connected the CXL Endpoint (which includes terabytes of memory) to its CXL GPU using two MCIO (multi-channel I/O) cables. These high-speed cables support both PCIe and CXL standards, allowing for efficient communication between the GPU and memory.
However, adoption may not be straightforward. GPU cards may require additional PCIe/CXL-compatible slots, and significant technical challenges remain, particularly with integrating the CXL logic and subsystems into today’s GPUs. Integrating new standards such as CXL into existing hardware involves ensuring compatibility with current architectures and developing new hardware components, such as CXL-compatible slots and controllers, which can be complex and resource-intensive.
While Panmnesia’s new CXL-GPU prototype potentially promises unprecedented memory expansion for GPUs, its reliance on the emerging CXL standard and need for specialized hardware could create hurdles to immediate widespread adoption. Despite these hurdles, the benefits are clear, especially for large-scale deep learning models that often outstrip the memory capacity of current GPUs.
