Token cost
AI token cost is the charge associated with the units of text or other content that an AI model processes and generates. Providers may price input, cached input, and output tokens at different rates. Token cost is a major input to the cost of many large language model applications, but it is not the same as the total cost or value of the work produced.
What is a token?
A token is a unit a model uses to process content. It may be a whole word, part of a word, punctuation, or another encoded element. One word does not always equal one token, and tokenization varies by model and language. OpenAI distinguishes input, output, cached, and reasoning tokens, and notes that actual usage depends on the model and the text being processed (OpenAI Help Center).
That is why estimating cost from word count alone is unreliable. Use the tokenizer or usage data supplied for the actual model whenever possible.
How is AI token cost calculated?
A basic calculation is:
Token cost = (input tokens / 1,000,000 × input rate) + (cached input tokens / 1,000,000 × cached rate) + (output tokens / 1,000,000 × output rate) + other provider fees
Use the current pricing page for the chosen model because rates and billing categories can change. OpenAI, for example, publishes separate input, cached-input, and output rates for supported models (OpenAI API pricing).
An illustrative calculation
Assume a hypothetical model charges $2 per million input tokens and $8 per million output tokens. A run uses 10,000 input tokens and 2,000 output tokens. Its token cost would be:
(10,000 / 1,000,000 × $2) + (2,000 / 1,000,000 × $8) = $0.036
This example only demonstrates the formula. It does not represent current pricing for a named provider or model.
An LLM cost calculator applies the same logic across a projected number of calls. A useful calculator should separate input and output volumes, account for cached input where applicable, identify the pricing date, and disclose which fees it excludes.
Why cost per token can be misleading
A lower price per million tokens does not guarantee a lower cost per completed task. A cheaper model may need longer prompts, more retries, additional tool calls, or more human correction. A more expensive model may complete the same work in fewer attempts. Context length, generated output, reasoning usage, retrieval, and application design all influence the final bill.
Cost per token is therefore best used as a component metric. Pair it with success rate, quality, latency, and a workflow measure such as:
Cost per accepted completed unit = total workflow cost / accepted completed units
This shows whether lower token spend actually lowers the cost of useful work. AI unit economics explains how to choose the unit and the cost boundary behind that measure.
How can teams control AI token costs?
Measure usage by model, product, team, and workflow first. Then identify the main sources of unnecessary volume. Common fixes include removing duplicated prompt context, limiting overly long outputs, caching repeated context, using retrieval to supply only relevant material, and routing simpler requests to a smaller model.
Do not trim context blindly. Missing instructions or evidence can create failures that cost more than the saved tokens. Test changes against a stable evaluation set and monitor accepted output, retries, escalations, and latency. AWS recommends combining token management with model selection, caching, and architecture choices rather than treating tokens in isolation (AWS cost optimization guidance).
Token cost versus inference cost
AI inference cost is the cost of running a trained model to produce an output. For a metered language-model API, token charges may make up most of that cost. For a self-hosted or provisioned system, inference cost can also include idle accelerator capacity, orchestration, storage, networking, and monitoring.
For a wider view of subscriptions, infrastructure, review, and rework, see AI cost optimization, and for the full lifecycle boundary see AI total cost of ownership. For the ownership and forecasting process around AI spend, see FinOps for AI and AI spend management.