// HACKER NEWS — CYBERSECURITY
Linux 7.3 improves performance when running out of vRAM
Earlier this year, I blogged about work I did to improve VRAM management
for games. Now, after many months of floating around in mailing lists, the kernel patches
are finally merged upstream and queued for Linux 7.3! Hooray!
To celebrate, let’s look a bit deeper at one sentence I wrote in my previous post:
[Games] should perform much more stable - as long as the game itself doesn’t
use more VRAM than you actually have.
So, one may ask: What if they do, in fact, use more VRAM than you actually have?
Typical expectations for this seem to be that once this happens you’re pretty much screwed.
Games will start crashing left and right, performance plummets to unplayable levels,
a good gaming experience becomes impossible.
But is that really just an unavoidable fact of life? What really makes running out of
VRAM suck so hard? And, most importantly: How can we make it suck as little as possible?
In theory, running out of VRAM should exclusively be a performance issue, not a stability one.
Support for overcommitting VRAM has existed for as long as GPU drivers have: If the driver
overcommits VRAM, you are generally allowed to request as much VRAM as you’d like,
and you’ll get as much as the kernel driver decides it can fit into the physical memory that exists on GPU.
On the performance side, the big-picture reason for bad performance when you run out of VRAM is fairly simple.
As soon as the game requests more VRAM than is physically present, some of the game’s memory will
have to be moved/evicted to CPU RAM instead. For the GPU, accessing CPU RAM is much slower than VRAM:
Not only is CPU RAM slower than a dedicated GPU’s VRAM in general, all memory accesses also
have to go over the PCI bus. The PCI bus adds latency and is typically also the limiting factor in bandwidth when
fetching from CPU memory.
Due to PCI speed limitations, there are some truly unavoidable performance constraints when overcommitting VRAM.
Assuming the GPU is hooked up via a PCIe 4.0x16 connection, you get a little less than 32GiB/s of bandwidth. Each
millisecond, that PCIe bus can transfer ~32.2MiB of data. For a minimum framerate of 30 frames per second (33.3ms
per frame), the absolute maximum amount of data the GPU is able to access is ~1,075.5MiB, a tiny bit over 1GiB of
data. In other words, if so much memory gets evicted that the GPU needs to fetch more than 1GiB from evicted memory
in one single frame, it is simply impossible to still hit 30 FPS.
At the same time, just reading a little bit of CPU memory on the GPU is not immediately a death sentence for performance.
In fact, GPU drivers sometimes decide to let things like command buffer data and related allocations live in CPU
RAM even when there’s plenty of VRAM available! Whenever the GPU executes these commands, it has to access CPU memory, and yet
in these cases everything runs completely fine. So what makes these accesses different - why are they fine and yet
running out of VRAM seems catastrophic?1