Introduction
Content Filter covers the safety categories every organisation ends up needing and nobody wants to define from scratch.
Unlike most checks it is not one detector but seven, each ticked and tuned separately — which makes it the check where configuration matters most.
What it detects
"Common safety categories — hate, violence, sexual, self-harm, and more."
Category | Covers |
| Hateful content directed at people or groups |
| Violent content and threats |
| Sexual material |
| Self-harm and suicide content |
| Targeted abuse |
| Facilitation of criminal activity |
| Coarse language |
Thresholds
Tick each category to check, and set its threshold to High, Medium or Low. A category you leave unticked is not evaluated. The threshold is a confidence level, and the product states the trade-off directly: higher settings mean higher likelihood and fewer false positives.
Level | Behaviour | Use when |
Unticked | The category is not evaluated | It is not a risk for this agent, or another control covers it |
| Fires on weaker signals — more catches, more noise | Missing something is much worse than a false positive |
| Balanced | The sensible default for most categories |
| Fires only on strong signals | The category is noisy on your traffic and you want only clear cases |
Set each category separately. The instinct is to pick one level and apply it across all seven; the result is a control that is too loose where it matters and too tight where it does not. A customer-support agent may want Self-harm on Low and Profanity on High — missing a distress signal is serious, and flagging every swear word is not.
"Seven dials, not one. Turning them all to the same number is the fastest way to a control nobody trusts."
Where to run it
Applied on | Catches |
| What users bring to the agent — including distress signals worth routing to a human |
| What the agent says back. The reputational surface. |
Content Filter runs on User input and Agent output only, and you choose a Classifier model for it. Most teams start on Agent output — what your agent says is what you are accountable for. Add User input when you also want to know what is being asked.
Tuning
Tune per category, per agent — A clinical agent and a marketing agent need different settings on the same seven dials.
Watch Profanity — It is almost always the noisiest, and almost always the least consequential. Start it at
High, or leave it unticked.Treat Self-harm as an operational signal, not just a policy one — A hit on user input may need a human, not a log entry. Decide the escalation path before enabling it.
Grade the severity — These categories do not all matter equally. A single ungraded control across all seven flattens that.
Known limitations
Categories are fixed — You tune thresholds, not definitions. Something outside the seven needs
Denied TopicsorCustom (LLM check).One action across all categories — The control has one action. To block hate and only flag profanity, use two controls.
Context is not intent — Content quoting or reporting unsafe material can read as producing it.
It calls a model — Each evaluation uses the
Classifier modelyou choose, which lands in governance spend.