AI costs: calculate, compare and control inference spending

Calculate AI API costs from tokens and other usage, estimate a monthly workload, find cost per customer and reconcile the result to your provider bill.

By Keyveve, Inc. · Updated 2026-10-23

AI costs depend on what you run, how often you run it and how the provider bills it. For an application using a model API, start with inference usage: input and output tokens, cached tokens, or units such as images and audio minutes. Then add the infrastructure and services needed to deliver the feature.

This guide focuses on AI inference costs for applications and agents. Training a model, operating a GPU fleet and paying for a consumer chatbot subscription are different cost problems. A subscription price is not a per-request API rate.

Calculate the cost of an AI request

For a text model billed per million tokens, separate each billing category before multiplying by its rate:

request cost = (
    uncached input tokens × input rate
  + cached input tokens × cache-read rate
  + cache-write tokens × applicable write rate
  + output tokens × output rate
) / 1,000,000

This formula assumes those counts are disjoint. Provider usage fields have different meanings: a total input count may already include cached input. Subtract the cached portion before applying the uncached rate in that case. Do not add the same tokens twice. Cache-write durations can have different rates; Anthropic's cache documentation describes its write, read and expiration rules.

Use the rate for the actual provider, model version, service tier and region. Batch pricing, long-context thresholds, negotiated discounts and tool charges can change the result. Reasoning tokens may be billed as output even when they are not visible in the final answer. Images, audio and video may use separate units or token rules: a text-only formula is not a complete estimate for those requests.

Browse model prices, compare models, or use the AI cost simulator for a workload with multiple models. Each model page includes the price-book date and provider source so you can check its assumptions.

A worked monthly estimate

Suppose a feature makes 100,000 requests a month. Each request uses 2,000 uncached input tokens and 500 output tokens. For illustration, assume rates of $2 per million input tokens and $8 per million output tokens. These are hypothetical rates, not a quote for a named model.

If half the input tokens become cache hits at an assumed $0.20 per million read tokens, the input cost becomes $200 + $20. The total becomes $620, before any cache-write charge. That is a $180 reduction under these assumptions; it is not a promised saving. Cache hit rate and write cost determine whether caching pays off.

Add separately any retrieval, storage, web-search, compute, networking and monitoring charges. Avoid double-counting a tool charge already included in the provider total. A monthly estimate should show a baseline and a high-usage case rather than a single precise-looking number.

Why agents can cost more than a chat message

One user task can trigger several model calls: planning, retrieving context, checking a result, retrying and summarizing. Conversation history can make later calls larger. Measure cost per completed task, including failed attempts, as well as cost per model call.

Record a task or feature identifier alongside each call. Measure the distribution of calls per task and output length rather than assuming all tasks behave like the average. Set limits on iterations and output, and test the completion rate when changing those limits. A cheaper model that needs many retries may cost more per successful task.

Reduce costs without losing the result you need

1. Measure first. Identify the models, features and customers responsible for spend. Separate a traffic increase from a price change or a larger prompt.
2. Remove repeated work. Reuse suitable results and reduce unnecessary context. Distinguish an application cache hit, which avoids a provider call, from a provider prompt-cache hit, which still has a billable request.
3. Evaluate smaller models. Compare task success, latency and cost on representative inputs before switching production traffic.
4. Batch work that can wait. Check the provider's supported models, discount and completion window instead of applying a blanket discount.
5. Control expensive paths. Limit retries and agent loops; use customer budgets and alerts. Blocking requires an enforced check in the request path, not just a dashboard threshold.

Veqen's budget documentation explains alert and block behavior. The free-tier abuse guide covers per-customer limits.

Turn a provider bill into cost per customer

A provider invoice generally cannot identify which of your customers caused each call. Attach a stable customer or project ID at the application boundary, and carry it through model calls and retries. Prefer pseudonymous IDs over names or email addresses.

Then calculate:

customer inference cost = sum of recorded costs tagged to that customer
inference margin = (customer revenue - customer inference cost) / customer revenue

Label that result inference margin if other delivery costs are excluded. It is not automatically your company's gross margin. For a fuller calculation, include the applicable costs of serving that customer and handle zero revenue explicitly. See cost per customer and AI gross margin.

Reconcile estimates with the bill

Recorded cost and billed cost answer different questions. A versioned price book estimates a call from measured usage. A provider cost report records charges under that account's billing rules. Differences can reveal missing calls, custom rates, credits, timing boundaries or fees that request telemetry did not capture.

Compare the same provider account, currency, time zone and billing period. Preserve both numbers and investigate the difference instead of overwriting one to make them match. Credits and prepayments also affect cash and accounting differently from gross usage. See reconciliation, AI credits and accounting for AI costs.

Choose the next step

• Estimating an idea: use the simulator and vary volume, token counts and model choice.
• Comparing rates: inspect provider and model pricing, including cache, batch and non-token units.
• Running an application: follow the SDK quickstart to attach costs to customers and features.
• Working in an AI assistant: connect the MCP server. Public pricing tools need no account; private ledger tools require authorized access.
• Closing the books: start with provider reconciliation and the finance workflow.

Veqen's publisher is Keyveve, Inc. Its open-source SDKs calculate costs locally; the hosted ledger adds attribution, reconciliation and budget workflows. Check product pricing for plan limits. No estimator can guarantee the final invoice without the provider's billing data.