Expanding Core-ESXi RAM from 128GB to 593.72GB Using Samsung 970 EVO Plus NVMe SSD

Posted on

With the release of vSphere 8.0 Update 3, VMware introduced Memory Tiering over NVMe as a Technical Preview. This feature enables users to expand memory capacity on a host using locally installed NVMe devices. Several experts, including William Lam, have already tested this feature, and it appears to be working seamlessly.

Like many other home lab enthusiasts, I was excited to try this out. My Core ESXi server is a low-power machine that hosts essential infrastructure VMs like DNS servers, monitoring systems, vCenters, and more. Initially, the server came with 128GB of RAM, which was the maximum available at the time of purchase. Upgrading to 256GB is now possible, but it’s costly, and I’d rather invest in my AI/ML server than in this older setup—especially since the older RAM would be difficult to repurpose elsewhere.

To take advantage of the new memory tiering feature, I’m using my Samsung 970 EVO Plus 500GB NVMe SSD (with speeds of 3500 MB/s read and 3200 MB/s write) as my RAM drive. Here’s a simple guide on how to set it up.

NB! According to William Lam, not all CPUs are supported yet. The CPU must have vMMR capability. Follow his blog to find out when others will be supported.

Here’s how you can check if your CPU supports it:
Check if your CPU has the following:
Intel® Virtualization Technology (VT-x) Yes
Intel® Virtualization Technology for Directed I/O (VT-d) Yes
Intel® VT-x with Extended Page Tables (EPT) Yes

If your CPU has all three, then it should work.

How to Set Up Memory Tiering Over NVMe:

  • Log in to the ESXi Server
  • Navigate to Storages and Devices
  • Under Devices, choose the NVMe drive you want to use for tiering
  • Note the Path, such as:
  • /vmfs/devices/disks/t10.NVMe____Samsung_SSD_970_EVO_Plus_500GB__________7XXXXXX
  • Copy this to Notepad for later.
  • Enable ESXi Shell and SSH
  • Use Putty to SSH into your ESXi host.

Run the following commands:

Enable Memory Tiering:

esxcli system settings kernel set -s MemoryTiering -v TRUE

List storage devices if needed:

esxcli storage core path list

Verify the drive has no partitions:

partedUtil getptbl /vmfs/devices/disks/t10.NVMe____Samsung_SSD_970_EVO_Plus_500GB__________7XXXXXX

If partitions exist, delete them:

partedUtil delete /vmfs/devices/disks/t10.NVMe____Samsung_SSD_970_EVO_Plus_500GB__________7XXXXXX 1

Create the memory tier device:

esxcli system tierdevice create -d /vmfs/devices/disks/t10.NVMe____Samsung_SSD_970_EVO_Plus_500GB__________7XXXXXX

Verify the tier device creation:

esxcli system tierdevice list

Adjust the tier percentage to 400% (this value can vary based on your setup):

esxcli system settings advanced set -o /Mem/TierNvmePct -i 400

Reboot your ESXi host, and you’re done!

This method allowed me to boost my core server’s RAM from 128GB to 593.72GB using the Samsung 970 EVO Plus 500GB NVMe SSD. It’s a cost-effective solution that saves me from expensive hardware upgrades and extends the life of my existing server while keeping my core infrastructure running smoothly.

I hope this guide helps other home labers looking to do the same!