// TOM'S HARDWARE US — HARDWARE & GADGET
Benchmarking Qwen 3.8 27B on RTX 5090 and beyond — VRAM capacity alone can't overcome severe software and inference engine bottlenecks
When you purchase through links on our site, we may earn an affiliate commission. Here’s how it works.
Alibaba’s Qwen 3.8 27B open-weight AI model came out a couple of weeks ago, and it immediately created a wave of hype among local AI enthusiasts thanks to its impressive intelligence benchmark results for a model of its size and capabilities.
Totaling around 17GB for four-bit quantized weights and offering built-in multimodal capabilities on top of its general aptitude, Qwen 3.8 27B immediately grabbed the attention of everybody with an RTX 5090, RTX 4090, or RTX 3090 (as well as a Radeon RX 7900 XTX, Radeon AI Pro R9700, or Arc Pro B70).
Were we on the verge of frontier-level intelligence from a four-bit quant on a single graphics card? Could everybody with a capable enough local AI setup go and cancel their Claude or ChatGPT subscriptions?
The answer, of course, as with every open-weight AI model hype cycle, is more complicated than just eyeballing the size of the model weights and comparing it to your available VRAM pool. Does the card or system you're using to host the model have enough VRAM left over to provide useful amounts of space for the model's context once everything is running? Do your host system and LLM inference engine deliver acceptable time-to-first-token, as well as high throughput beyond just bench-racing from an empty context window?
It's one thing if you just want to chat with a model and see what happens; it's another entirely if you want to put it to work, especially as impatient agents take the limits of human perception out of the picture.
We wanted to see what hardware and software stack Qwen 3.8 27B really wants in order to deliver solid performance, so we ran it on systems ranging from a desktop PC with discrete GPUs to systems with unified memory architectures like the DGX Spark, Mac Studio, and Ryzen AI Halo.
Our discrete GPU AI testbed includes the following components:
Where it was possible to do so, we tested performance with Qwen 3.8 27B’s built-in multi-token prediction capabilities both enabled and disabled. Not all of the model runners we tested were able to support MTP within the amount of VRAM available to us on some of our platforms. We note where MTP was and wasn’t possible in our analysis of each platform, as well as in our charts.
We started with the RTX 5090, whose 32GB of GDDR7 and 1.8 TB/s of memory bandwidth would seem to make it an absolute no-brainer for getting the best local inference performance with this dense model. (Mixture-of-experts models tend to be friendlier to performance on lower-end hardware like the DGX Spark and AMD's Strix Halo, as their limited numbers of active parameters mean less data movement during inference).