Haystack cost tracking

Record the cost of each Haystack chat generation with a pipeline component, attributed to a customer and feature.

By Keyveve, Inc. · Updated 2026-10-23

A Haystack component records the cost of each reply from an OpenAI-compatible chat generator. Connect it after the generator; it passes the replies through unchanged.

Install

pip install "veqen[haystack]"

Add the component

from haystack import Pipeline
from haystack.components.generators.chat import OpenAIChatGenerator
from veqen.integrations.haystack import VeqenOpenAICostTracker

pipeline = Pipeline()
pipeline.add_component("llm", OpenAIChatGenerator(model="gpt-4o-mini"))
pipeline.add_component("cost", VeqenOpenAICostTracker(tags={"feature": "assistant"}))
pipeline.connect("llm.replies", "cost.replies")

result = pipeline.run({"llm": {"messages": messages}, "cost": {"customer": org.id}})

The component's outputs say whether the call was recorded (recorded) and why not when it wasn't (recording_status). For another OpenAI-compatible host, pass provider="together" (or the host's name) so the model is priced at that host.

Limits

It records after the reply, so it doesn't stop a call that would go over a budget. Use one reply per request (n=1): several alternatives can repeat the request's total usage, so they are not summed. Needs Python 3.11 or later.

Next

• Python SDK: wrap() the client inside your own components instead.
• Tags: what each tag is for.