// TOM'S HARDWARE US — HARDWARE & GADGET
A primer on key AI LLM concepts — KV cache, weights, quantization, and how inference actually works
Everyone knows about hallucination, but a handful of topics are often unclear. We'll fix that.
When you purchase through links on our site, we may earn an affiliate commission. Here’s how it works.
As hardware for running local AI models continues to move to the forefront, such as Nvidia's RTX Spark, a new set of acronyms, definitions, and terminology is popping up. Given the pace of releases, we've put together a list of AI's new nomenclature to familiarize you with new concepts and pieces of terminology that you'll likely see in Tom's Hardware Premium articles.
This primer should get you started on the world of AI and be handy the next time news on the topic comes around, and possibly when you're downloading some models to try out and tweak for yourself. For the context of this article, we're going to stick with notions related to text LLMs, or in common parlance, chat, programming, and most agentic bots.
This is probably the easiest one: a model is the LLM itself (a Large Language Model), computed from training sets of mostly text data on the scale of terabytes to petabytes. That training information comes from the internet, books, video and audio transcripts, and other sources.
The model is roughly analogous to a set of highly advanced layers of statistical analysis of relationships between tokens — words or parts of words, computer code, punctuation, and other symbols. The tokens themselves aren't part of the model, but they are packaged along with it. An AI model generates tokens in its outputs.
Those computing layers within an AI model would be useless without corresponding weights, which represent how strong the connection is between sections of tokens, sort of like how a pathway between neurons is more traveled than others. When the model runs, these values end up establishing how tokens end up relating to each other.
Example: the weights of the connections between "na" and each next part for "naval", "nachos", or "naruto" will all be different, and the higher the weight, the more often that word appears in training data. Importantly, weights cannot change without re-training the model, or parts of it.
When you see LLM specs online, you'll see something like "4B", "8B+", "20B", and so on and so forth. That shows how many parameters, or weights, the model uses, and it's generally measured in billions for consumer-grade models, and trillions for frontier models like GPT, Claude, and Kimi.
Note that a model having more weights than another doesn't automatically mean it's smarter. Just like with CPU clock speeds, it's not just about getting a higher number — bigger is usually better, yes, but you want those weights to be meaningful for the output.