AMD's new trick lets servers use half as much DRAM with minimal performance loss

AMD has found a way to reduce server memory costs: its Mext technology moves less frequently used data from DRAM to much cheaper SSDs, while AI algorithms predict which data applications are likely to need next to minimize the performance impact. The technology is already running on 145 systems, with estimated savings of around $5 million in memory costs.

AMD acquired Mext in June and is now preparing to bring the technology to enterprise customers running Epyc processors. According to Gary Smerdon, Mext's former CEO and now AMD's corporate vice president of business development, memory can account for 60 % to 90 % of the cost of large servers. Replacing some of that DRAM with SSD storage could therefore save customers a considerable amount of money.

The idea is similar to swap files or the Windows page file, which move data from RAM to storage when memory is running low. The problem is that flash storage can be up to 500 times slower than DRAM, so this approach usually comes with a performance penalty.

Mext aims to avoid much of that slowdown by using machine learning to predict which memory pages applications will need and move them back into RAM before they are requested. Smerdon cited Meta research showing that 50 % to 80 % of memory pages had not been accessed for at least five minutes. This suggests that a substantial amount of data occupying expensive DRAM may not be actively used.

The software uses several machine-learning algorithms, including a model derived from transformer technology, to predict future memory requests. AMD says initial training takes less than a second on a single Epyc CPU core. The model then adapts as workloads change, without requiring a GPU.

AMD's tests show that Mext's results depend on the workload and how much DRAM is replaced with flash. In tests with Synopsys PrimeTime, an electronic design automation tool, Mext replaced half of the system's DRAM while retaining 95 % of its original performance. When DRAM made up just 25 % of the memory and flash accounted for the remaining 75 %, performance dropped to 80 % of the original level.

AMD reports better results with Apache Spark, where Mext retained 99 % of performance after half of the DRAM was replaced with flash. The company says the technology is already running on 145 systems. Smerdon estimates that customers have saved around $5 million by avoiding additional memory purchases.

There are some limitations, however. In virtualized environments, the software requires one CPU core per virtual machine or virtual private cloud instance. That could create significant overhead on servers running many virtual machines.

AMD plans to license Mext through its OEM partners, and the software will work exclusively with AMD hardware. The company is working with manufacturers such as HPE to prepare compatible Epyc systems. It is also developing a separate RAM View application to help customers identify which servers and workloads could benefit from the technology. However, pricing and a launch date for wider availability have not been disclosed.