veqen
Sign in Start free

Runs and conversations

See what one agent run, conversation or batch job cost from start to finish, how run costs spread, and which runs cost the most.

An agent answering one question can make dozens of model calls: planning, tool calls, retries, a summary. Tag them with the same run and Veqen adds them up, so you can see what each run cost, what a typical one costs, and which ones ran away.

Tag a run

Any id that's the same for every call in the run works: a conversation id, a job id, a trace id.

with veqen.tags(run=conversation.id, customer=account.id):
    answer = agent.run(question)

await withTags({ run: conversation.id, customer: account.id }, () => agent.run(question));

The OpenAI Agents SDK processor tags each agent run with its trace id for you. Over OpenTelemetry, veqen.run, gen_ai.conversation.id or ai.telemetry.metadata.run set it. Ids are up to 128 characters; send them on every call of the run, including retries.

What you see

Agent runs shows, for the dates you pick:

  • how many runs ended, and what they cost in all;
  • a typical run (the median: half cost less, half more), and the 90th and 99th percentile runs;
  • how run cost spreads, in bands from cents to dollars, so a long tail stands out;
  • the most expensive runs, each opening to its calls in order, with the model, tokens and cost of each step.

Filters from Breakdown (a customer, a feature) carry over, so you can compare one customer's runs with everyone's. A run's page shows its last 31 days of calls if the same id is reused for longer.

Use it

  • Price an agent. The median and p90 run cost are what you need to set a per-task price or a plan limit.
  • Find runaway loops. A run in the top band that made hundreds of calls is usually a tool loop or a retry storm.
  • Watch a change. Compare the run spread before and after a prompt or model change.

With a full-access key, GET /v1/runs returns the count, total, average, typical (median_about) and 90th percentile (p90_about) run and the most expensive runs, and GET /v1/runs/{run} one run's calls in order; over MCP, get_runs and get_run.

Runs are counted as calls arrive, so they're current to the last ingest. The run id isn't a dimension in Breakdown; filter by customer or feature there and open Agent runs for the per-run view.