Energy Per Inference Console

Every token a language model generates costs a measurable amount of energy, and at the scale AI now operates — trillions of tokens served to a global user base — that tiny per-token figure compounds into the dominant cost and carbon line of the whole enterprise. Understanding it starts with a single, exact relation: energy per token is the serving hardware's power divided by its throughput in tokens per second, because a watt is just a joule per second.

That number is the right efficiency metric precisely because it unifies the two things that pull against each other. Raw tokens-per-second flatters a power-hungry system; raw wattage ignores the work done. Energy per token captures throughput-per-watt, so a fast-but-thirsty accelerator and a slower-but-efficient one are judged on the same honest basis — and it's expressed in tokens, the unit inference actually bills in, so it flows directly into cost per million tokens and carbon per query.

The strategic insight is that inference, not training, dominates a model's lifetime energy. Training is a single large expense; serving is billions of queries over months and years, and the cumulative energy overtakes training many times over. That's why the levers that raise throughput-per-watt — batching, quantization, efficient hardware, speculative decoding — matter so much: each one lowers energy per token, and therefore cost and carbon per query, simultaneously. The sensitivity bars here show that leverage directly.

Because cost per million tokens is the unit of the AI economy — the floor under API pricing and the basis of capacity planning — this console reports it in your chosen currency, tied to real power and throughput. Scale a single node's figure to a full fleet in the Data Center Power console, and remember to apply PUE for the all-in facility cost.