Token Cost Console

Every LLM deployment eventually faces the same question: keep paying an API per token, or stand up your own serving? The answer lives on a single axis — cost per million tokens — and the honest comparison puts your self-hosted figure next to the API's menu price on that exact axis. Self-hosted cost is disarmingly simple: the all-in hourly cost of the serving GPU divided by the tokens it actually produces in that hour.

That denominator is everything. A GPU's hourly cost is roughly fixed whether it's flat out or idle, so cost per token is inversely proportional to utilization and throughput. Run it hard, batching many requests, and the cost per token drops to a fraction of any API price. Run it at twenty percent, and each token costs five times as much — often worse than the API. This is why self-hosting is a high-volume game and the API is the right answer for everyone below the threshold.

The API's price is higher per token for good reasons: it bundles the provider's margin, their infrastructure and operations, and the genuine value of zero setup and instant elastic scale. Below a break-even volume, that bundle is a bargain — you'd waste far more on underused hardware. Above it, you're paying the markup on billions of tokens, and the raw economics of owning the serving dominate. The break-even, which this console computes, is where fixed operating costs and the per-token gap balance.

So the decision is really about your steady volume and how hard you can drive utilization. Push throughput up with batching — modeled in the LLM Serving console — to lower self-hosted cost per token, and decide whether to own or rent the GPUs in the Accelerator ROI console.