Calculate prompt caching savings without double-counting tokens

Separate fresh input, cache reads, cache writes and output to measure actual prompt caching savings and check the result against your provider bill.

By Keyveve, Inc. · Updated 2026-10-23

Cached tokens are not generally free. A useful caching estimate separates the tokens that were actually read from cache from fresh input, cache writes and output. The provider's usage response is the starting point; repeated prompt text alone does not prove a cache hit.

Calculate one request

For rates quoted per million tokens:

cost = (fresh_input × input_rate
      + cache_reads × cache_read_rate
      + cache_writes × cache_write_rate
      + output × output_rate) / 1,000,000

This is a simplified calculation. Service tiers, long-context thresholds, residency and different cache-write lifetimes may change the applicable rates. Use the model's rate table and price-book rules, then compare to the provider bill.

Consider an illustrative workload with 2,000 fresh input tokens, 8,000 cache-read tokens and 500 output tokens. Suppose the rates are $3 input, $0.30 cache read and $15 output per million tokens:

At the same hypothetical rates, treating all 10,000 input tokens as fresh would cost $0.0375 including output. The read saves $0.0216 on this request, about 58% of the total. That is not the same as a 90% discount on the entire request. These numbers illustrate the method, not a current model quote.

Avoid the two common mistakes

First, check whether a provider's reported input total already includes cache reads. OpenAI-shaped prompt totals include cached input, so subtract the cached count before passing fresh input separately. Veqen's wrappers handle supported provider usage formats for you.

Second, count cache creation. A reusable prefix may have a write charge before later reads save money. If the extra write cost is W and each reuse saves S, the cache needs more than W / S reuses to produce net savings. Failed hits, expiry and changing prefixes can reduce that benefit.

import veqen

# Values from the provider's usage response, not tokenized prompt strings.
cost = veqen.cost(
    "gpt-4o",
    input_tokens=2000,
    output_tokens=500,
    cache_read_tokens=8000,
)

This uses the locally installed SDK's price book. The hypothetical rates above are separate from this real model lookup. See Python pricing for supported cache-write and service-tier arguments.

Measure by feature and customer

A site-wide cache-hit average hides the product changes that matter. Split cost and cache usage by feature, then by customer or project. A common system prompt may cache well while customer-specific document retrieval changes on every request. Preserve useful context; do not sacrifice answer quality just to improve the hit rate.

Veqen's Savings and Insights can identify caching opportunities from recorded usage. Estimates describe an opportunity, not a provider guarantee. Use the simulator for a workload with explicit fresh and cached inputs, and compare actual cost before and after a controlled change.

Check the financial result

Use the same date range and billing host when comparing the request ledger to the provider's bill. Credits, negotiated discounts, untracked traffic and delayed reports can explain differences. A cache estimate is not proof that an invoice decreased by that amount.

Next: cost per customer, agent cost per run, and model pricing.