Compare self-hosting an open-weight LLM against a commercial API. Published GPU rental rates, real model architectures, and your break-even volume.
Start from a deployment
GPU hourly rates are on-demand prices published by Lambda and RunPod, read on August 23, 2026. Card memory and bandwidth come from NVIDIA. Rental prices change; check the provider before committing.
Costs cover GPU rental and API tokens only — not engineering time, storage or egress. To check whether a model fits a card before pricing it, use the GPU memory calculator.
You might also find these calculators useful
Pick your GPU and see which LLMs actually fit
Monthly LLM API bill with caching and batch discounts
Parameters, VRAM and GPU fit from a model's real architecture
Price a fine-tuning run on published vendor rates
Renting a GPU by the hour and calling a commercial API are priced in different units, so the two are hard to compare by eye: one bills for time you hold the card, the other for tokens you send. This calculator puts both on the same footing. Pick an open-weight model, a rentable GPU and the volume you expect, and it returns the monthly cost of each, the cost of a single request either way, and the daily volume at which self-hosting overtakes the API.
A self-hosted deployment costs the same every hour whether or not requests arrive, so its cost per request is set by how much of the card's capacity you actually use. Capacity in turn comes from memory bandwidth: generating a token requires reading every active parameter plus the attention cache, so a card can emit at most its bandwidth divided by those bytes. An API inverts this entirely, charging per token with no idle cost and no capacity to plan. The crossover falls wherever the API's price per request exceeds what one more GPU costs divided by the requests that GPU can serve.
Cost per request, self-hosted
A team already paying per token checks whether their current volume justifies moving to rented GPUs, and what volume would.
An engineer works out how many cards a model needs at their concurrency and context length, and what that costs per month.
A product owner puts a per-request cost on an AI feature under both options while the design is still changeable.
A developer compares the same model at bf16 and int4 to see how much the card count, throughput and monthly cost move.
Every hourly rate is a figure published by Lambda or RunPod, shown with the provider, board variant and tier it was quoted for, so you can open their page and confirm it. Where nobody publishes a rate for a card, the card is not offered rather than given an estimated one.
Self-hosting does not beat an API at every volume, because adding requests eventually adds GPUs. When the marginal cost of another card exceeds what the API charges for the same requests, the calculator says self-hosting never breaks even instead of quoting a threshold you would never reach.
Tokens per second comes from the card's published memory bandwidth and the model's active parameter count, not a lookup table. It is stated as a ceiling, because a real serving stack lands below the roofline and nothing lands above it.
VRAM comes from the same engine as the Model Size and GPU Memory calculators, so a configuration that will not fit is reported as needing more cards rather than being quietly costed as if it ran.
From the published on-demand price pages of Lambda and RunPod, both of which quote per GPU per hour — the unit this calculation needs. Each rate is labeled with its provider, board variant (SXM and PCIe are different products at different prices) and tier. Rates for AWS, Google Cloud and Azure are not included: AWS prices whole eight-GPU instances, and the Google and Azure GPU tables are rendered in the browser rather than served, so neither can be verified the way the others can.
Because the list is limited to cards that a cloud provider actually publishes an hourly rental rate for. Most consumer cards are not rented by the hour at all, so quoting a price for one would mean inventing it. If you own the hardware or have a negotiated contract, switch the rate to custom and enter what you pay.
No — it is an upper bound. Each decoding step must read the active weights and the attention cache from memory, so throughput cannot exceed memory bandwidth divided by those bytes. Real serving stacks fall short of this because of prefill, scheduling, sampling and kernel overheads. Treat it as the ceiling your setup is measured against, and use a custom rate plus your own measured throughput if you have benchmarked.
That growing your volume will not change the answer. Past one card's capacity, more requests mean more cards, so the self-hosted cost rises roughly in step with volume rather than being spread over it. If the API already charges less per request than one more card costs divided by the requests it serves, no volume closes the gap. Cheaper hardware, higher concurrency or heavier quantization can, because all three lower that marginal cost.
A decoding step reads the model weights once and produces one token for every sequence in the batch. More sequences therefore mean more tokens for nearly the same memory traffic, which raises total throughput and cuts cost per request. Each individual sequence still gets only its share of that throughput, so any one response takes longer to finish. Batching trades per-request latency for cost.
No. It compares the rental rate of the GPUs against API token charges. Rented instances already bundle power, cooling and the host machine into the hourly rate, but engineering time, model storage, egress and the cost of running a serving stack are not included. Those fall on the self-hosted side, so a narrow self-hosting win is narrower than it looks.
Serverless bills for the seconds your code actually runs, or per token, rather than for holding a card. That makes it a different pricing model rather than a discount or premium on an hourly rate, and it is why this calculator does not offer it as an option. The figure to watch if you are considering it is utilization: the lower your utilization, the more you are paying for an idle card and the more attractive per-second billing becomes.