Pick your GPU and see which open-weight LLMs fit in its VRAM at your context length, which need quantizing, and how many cards the rest take.
Memory comes from each vendor's own product page. Pick the card you own, or enter a memory figure for anything not listed.
You might also find these calculators useful
Parameters, VRAM and GPU fit from a model's real architecture
Compare self-hosted GPU vs API inference costs
See whether your prompt fits an LLM context window
Price a fine-tuning run on published vendor rates
Most VRAM calculators start from a model and tell you how much memory it wants. This one starts from the card sitting in your machine. Pick it, say how long a context you need, and you get the whole open-weight catalogue sorted into what runs, what runs once quantized, and what would take more cards than you have. Memory comes from each vendor's own product page and every architecture is read from the model's own configuration file, so the parameter counts are derived rather than looked up.
Three things share your VRAM. The weights are fixed once you choose a precision: parameters multiplied by bytes per weight. The KV cache grows with context length, and it is the term people underestimate, because it scales with the key and value heads rather than the query heads. Grouped-query attention shrinks it dramatically, so a modern model at 32K tokens often needs less cache than an older one at 8K. Last comes the workspace the runtime needs for live tensors. On top of the three, allow around a fifth again for fragmentation and kernels, which is what stops a model that fits exactly on paper from loading at all.
What has to fit in VRAM
Someone comparing a 16 GB card with a 24 GB one runs both through and sees exactly which models cross over, rather than guessing from a parameters-per-gigabyte rule.
A developer about to pull a 40 GB checkpoint over a slow connection checks first whether it will load at all, and at which precision.
An engineer picking between an L40S and an A100 80GB compares what each one holds at the context length the product actually needs.
A team hitting out-of-memory errors at long context reads off the longest context their card can hold for that model, and decides whether to quantize or shorten the window.
Nobody chooses a model and then goes shopping. You own a card, or you are choosing between two, and the real question is what each one unlocks. A 12 GB card and a 24 GB card differ by one quantization step in practice, not by twice the capability.
A model that overflows at BF16 usually fits at INT8, and almost always at INT4. Rather than making you try each setting, the table names the widest precision that fits on the hardware you described.
Weights are a fixed number you can look up. The cache is not, and at long context it can exceed the weights. The longest-context column tells you where your card actually runs out for each model.
A 235B mixture-of-experts model holds all 235 billion weights in memory but reads only around 22 billion per token. That makes it enormous to load and quick to run, and no per-billion rule of thumb captures both.
At INT4, a 24 GB card comfortably holds a 32B dense model at a moderate context, and most 7B to 14B models with room to spare for a long window. At BF16 the same card tops out around 8B to 12B. Pick RTX 4090 above and the table gives the exact list rather than a rule of thumb, because the cache at your context length moves the boundary.
The speed ceiling needs a memory-bandwidth figure, and NVIDIA publishes one for every data-centre part and for the whole GeForce 50 series, but for nothing in the 30 or 40 series. The RTX 4090 product page prints the interface width and the memory type and stops there. Third-party sites quote a figure; it is not a vendor number, so this calculator leaves the column out rather than presenting someone else's arithmetic as a specification.
Card memory is sold in decimal gigabytes: a 24 GB card holds 24 billion bytes. Every figure on this page uses that same decimal gigabyte, so the requirement and the capacity can be subtracted directly. Most model tooling reports gibibytes instead, which are about seven per cent larger, so a model this page calls 20.4 GB shows as 19.0 GiB in llama.cpp or on a Hugging Face page. Same bytes, different unit — and mixing the two is exactly how a model that looked like it fit refuses to load.
Every expert has to be resident in VRAM, so the memory figure uses the total parameter count. Only the experts routed to each token are read during generation, so the speed ceiling uses the active count. That is why a 235B model with 22B active is both hard to fit and fast once it does.
Only roughly. The card count multiplies the memory available, which is right for a model split cleanly across cards, and the verdict column reports how many cards a model needs. It does not model interconnect overhead, per-card duplication of the cache, or the extra memory a tensor-parallel runtime holds, all of which make real multi-card setups slightly worse than the arithmetic.
Every architecture is transcribed from the model's own configuration file, or from the model's own technical paper where the repository is licence-gated, and the parameter count is derived from that architecture rather than taken from the name. A test checks each derivation against the size the model is named after. Card memory and bandwidth come from the vendor's product page.