Introduction

Outcomes is the part of the product most likely to produce a number somebody disputes in a meeting. Most of those disputes trace back to one of three things: the outcome names an activity rather than a result, the verification is weaker than the claim, or the spend denominator is not what people assume.

Nothing is being counted

1. Does the outcome have a source? — An outcome with no source is a definition with nothing measuring it. A source is an agent credited with producing the outcome.

2. Does the source have a check? — The product is explicit: "Each source needs its own check for when a session counts." No check, no counting.

3. Is the rule reading a signal the session actually carries? — A rule reads something already in the telemetry. If the signal is not emitted, the condition never matches — and it fails silently.

4. Is the agent sending traffic at all? — No sessions, no outcomes. Check the registry before the outcome configuration.

Note: A rule is "only as strong as that signal" — the product's own words. That is not a caveat about accuracy; it is a statement that a rule cannot see what the session does not record. If nothing in the trace marks success, no rule can find it, and the honest options are to emit the signal or to use a judge.

The count looks wrong

The ROI number is challenged

Expect this, and be ready with the three things that make it defensible:

1. Say what each counted session is worth and where the figure came from — A set amount should be the cost avoided each time the outcome happens. A figure nobody can source is a figure nobody will accept.

2. Say how the outcome was verified — A rule reading a hard signal and an AI judge grading output are different strengths of evidence, and the audience deserves to know which.

3. Quote the coverage alongside the return — Return on spend covers measured spend. Spend attributed to no outcome is real and excluded — see Reading the ROI View.

"A return-on-spend figure quoted without its coverage is not a strong claim. It is a strong claim about a subset, presented as though it were about everything."

The AI judge is expensive

It is a model call per session, and the product says so where you enable it: it "adds spend every session; captures what no signal records." That is the trade, stated honestly.

Three ways to keep it proportionate: use a rule wherever a real signal exists and reserve the judge for what nothing records; narrow the source's conditions so fewer sessions reach the judge; and check that the value of the outcome comfortably exceeds the cost of verifying it. An outcome whose verification costs more than the outcome is worth is measurement for its own sake.

Frequently asked

1. Rule, AI judge or Automation? — Rule if the session already carries the signal — no added cost. Automation if the system of record already knows — also free. Judge if neither does.

2. What threshold should a judge use? — The Pass score threshold defaults to 0.85. Treat it as a starting point and check a sample of decisions on both sides of it.

3. Can one agent be a source for several outcomes? — Yes, and it often should be — one agent can produce more than one kind of result.

4. What is Unmeasured spend? — Spend attributed to no outcome. The tile is there so the return is not read as covering everything; do not read past it.

5. Can I change what a session is worth later? — Yes, in the agent's What it is worth card — and say so when you do. A return figure that moves because the assumption moved needs that stated, or it looks like performance changed.

6. Who owns an outcome? — There is no Owner field; the outcome shows the person who created it. Agree the value with whoever will defend it in a meeting.

Related

• Outcomes & ROI

• Reading the ROI View

• Define an Outcome (LLM Judge)