Introduction
The library is a set of pre-built metrics covering the checks most agents need. Starting from one is almost always better than writing your own — not because writing is hard, but because a library metric has already been calibrated and yours has not.
Where to find it
Evaluation > Metrics > Explore Library. Search by name, and filter by test case type.
Note: This page does not list the library's contents. The library is maintained content that grows between releases, so a list here would go stale without anything failing. Browse it in the product; what this page covers is how to choose from it and what to check before you rely on one.
Filter by test case type first
A metric is built for either single-turn or conversational cases, and one written for the wrong kind will not score your dataset. Filtering by type before reading names saves working through metrics that cannot apply.
Reading a library metric before you adopt it
What parameters does it declare? — It scores only cases carrying all of them. A metric needing ground-truth context is useless against a dataset without it — see Input Parameters Reference.
What type is it? — An AI metric costs a model call per case. Over a dataset you run weekly that difference compounds.
How does it express a result? — Binary gives a verdict directly. A scale needs a threshold, and the threshold is your standard rather than the library's.
What is it actually measuring? — Read the description rather than inferring from the name. Metrics with similar names routinely check different things.
A sensible starting set
Rather than adopting many metrics at once, most teams converge on a small set answering distinct questions. In rough priority:
Question | Why it earns a place |
Did the answer address the question? | The most basic failure, and common |
Was it faithful to the supplied facts? | The hallucination check — needs ground-truth context |
Did it call the right tools? | The governance-relevant one, for agents that act |
Did it meet format requirements? | Deterministic and consistent, catches real breakage |
Four metrics answering four different questions beats eight overlapping ones — and costs half as much per run.
"Every metric you add is a model call on every case, forever. Add the fourth one only when it answers something the first three cannot."
Clone rather than write from scratch
Where a library metric is close but not right, cloning it gives you its structure, parameters and calibration as a starting point. That is nearly always faster and better than a blank editor — most of the work in a good metric is in the criteria wording, and a library metric's wording has been tested.
Verify before you trust it
A library metric is calibrated in general, not for your domain. Before it influences a decision, run it against a handful of cases where you already know the right answer and check that it agrees with you.
A metric that disagrees with your judgement consistently needs its criteria rewritten, not more data. See Create a Custom Metric.