Introduction
The control editor has a test tool sitting beside the form: paste sample content, run the check, see what it would have done. It is the difference between deploying a rule and guessing at one, and it is the step most often skipped.
Where to find it
Open any control for editing, or start a new one. The panel is on the right — Test this control, "Try sample inputs to see what it would do."
Before you have chosen a check type it says "Pick a check type to define the inputs this control reads." The tool cannot offer sample boxes until it knows what surfaces the check reads.
Running a test
Configure the check first — Pattern, topics, threshold, prompt — whatever the check takes. The test evaluates the configuration in front of you, not a saved version.
Select the inputs — One sample box appears per selected input, labelled for it. This is what makes the test honest: you are testing the check at the surface you chose.
Paste realistic content — Real traffic, not a textbook example. This is the whole value of the tool and the point at which most tests go wrong.
Run test — The result appears below.
Reading the result
Result | The product says | Meaning |
| "Nothing in your sample matched this control" | The check ran and found nothing |
| "{inputs} in your sample matched this control" | The check matched, and names which inputs did |
Naming which input matched is the useful part. A control on three inputs that only ever matches one of them is a control that should be on one input.
Note: The test tells you whether the check matches. It does not perform the action. Nothing is blocked or masked, and no incident is recorded. That is what makes it safe to run against a control set to
Blockbefore anything is live.
What to test — and the half everyone skips
Most people test that the control catches the bad thing. That is the easy half and it is not where controls fail in production.
Test | Answers | Usually |
Obvious violation | Does it work at all? | Done |
Obvious non-violation | Is it wildly over-broad? | Done |
Near-miss legitimate content | Will it fire on normal work? | Skipped |
Real traffic from your logs | What is the actual false-positive rate? | Skipped |
Obfuscated violation | Is it trivially evaded? | Skipped |
The near-miss is the important one. A secrets check that fires on any long alphanumeric string will pass every test you invent and then fire on every order reference in production. Find three pieces of legitimate content that look like violations and run those.
"Testing that a control catches violations tells you it is not broken. Testing that it ignores legitimate work tells you whether anyone will still have it switched on next month."
Test twice as hard before you set an action to Block
A false positive on Flag costs someone a minute of triage. A false positive on Block stops real work, and the person it stops has no idea why. The asymmetry is large enough to justify a rule:
Never ship a new control straight to Block — Run it on
Flagagainst live traffic first and read what it catches.Promote to Block only after the flag queue is quiet — Quiet means true positives only, not merely few incidents.
Test the near-misses again at promotion time — Traffic changes. The samples that were clean a month ago may not be.
If the test does not run
"Test automation isn't configured for this environment." means the testing capability is not available in the environment you are in, rather than anything being wrong with your control. Raise it with your administrator; the control itself is unaffected and will still work once saved.
What the test cannot tell you
Your real false-positive rate. A handful of samples is not a sample of your traffic. Only running on
Flagfor a period gives you that.Latency. The test says whether it matches, not what it costs the request.
Interaction with other controls. Controls are tested one at a time; in production several run over the same content. See Scoping & Precedence.