// TOM'S HARDWARE US — HARDWARE & GADGET
AMD shares how it can retain 95% of server performance with just half the DRAM, sidestepping memory shortages with MEXT
MEXT offloads cold memory pages to flash storage and serves them up with a predictive AI model.
When you purchase through links on our site, we may earn an affiliate commission. Here’s how it works.
In June, AMD overtook the predictive memory engine known as MEXT, which claims to offer upwards of 99% of the performance of a system with full DRAM capacity while running with as little as 25% of the physical DIMMs. The technology offloads cold memory pages to flash storage and uses an AI model to serve pages before they’re needed, and AMD says it plans to extend the tech across a wide swath of its IT infrastructure next year.
The pitch behind MEXT seems too good to be true. According to AMD and MEXT founder Gary Smerdon, the technology can cut the physical DRAM required in a server by 20% to 75% depending on the application and performance target. That missing memory still registers as installed DRAM to the OS. Instead of DRAM, however, MEXT uses relatively inexpensive flash storage to offload cold pages, flying them back in as they’re needed with an AI model. In some of AMD’s internal workloads, such as Synopsys PrimeTime, the company says it’s able to achieve 95% of the performance of a full server while running at 50% DRAM capacity with MEXT.
AMD has deployed MEXT internally on 145 servers, which it says will expand to 2,000 servers next year. In addition, the company has announced a partnership with HPE to bring MEXT to customers. Currently, we’re in phase 1 of the rollout, with some EPYC Venice servers carrying a “MEXT Ready” badge and a dedicated NVMe drive or bay for the offload. Phase 2 brings MEXT licenses out to these servers, as well as the RAM View application. AMD hasn’t said when Phase 2 will begin.
The AI-driven DRAM pricing surge impacts more than just gaming PCs, with enterprises also feeling the brunt of price increases, even if they aren’t running AI workloads. If MEXT is able to achieve what AMD claims, it could be transformative given the current market. Because of that, it’s worth understanding what exactly MEXT is, as well as what AMD claims it can do.
Although MEXT is exciting in concept, it’s obvious that AMD is taking a conservative approach to the rollout. Extraordinary claims require extraordinary evidence, after all. The applications of MEXT are currently narrow, though AMD says it’s expanding the workloads it’s using MEXT on with promising early results.
The basic setup of MEXT is simple in concept: cold pages go to flash, AI brings those pages back as needed, and it’s as if the memory was never missing in the first place. The main problem should be obvious. Flash is orders of magnitude slower than DRAM, so streaming pages back into memory from flash as they’re needed isn’t feasible. MEXT’s trick is using an AI model to predict the pages you’ll need. And, as long as the model is able to stay ahead by at least the transfer latency from the SSD, it’s as if the physical DRAM was installed the whole time.
This isn’t a crazy concept considering how a transformer-based LLM, which MEXT uses, operates. If you’re unfamiliar with the inner workings, this open-source visualization breaks down the concept of a transformer AI model cleanly. The foundational concept is next token prediction, which underpins the design of modern LLMs. Given a set of tokens, the LLM determines which token (word, portion of a word, etc.) is most likely to come next. It makes a prediction. In the case of MEXT, it’s predicting what page is needed next, not what word comes next.
Memory offloading to flash isn’t a new concept, but MEXT founder Gary Smerdon says that AI is the critical difference in this technology. MEXT itself is deployed as a Linux kernel module that works with any Linux distro. It uses an “ensemble of AI models,” with the primary transformer-based LLM driving the technology. And critically, it doesn’t come pre-trained.