- “We tried a couple jailbreaks.”
- “The model refused, so we’re safe.”
- “Don’t worry, our system prompt is strong.”
Those statements are not evidence.
If you want public trust, you need reproducible checks that connect to product behavior.
The 5 checks that build real evidence
1) Define what “pass” means
Before you run prompts, write one sentence per test:
- what must not happen (no secret leakage, no unauthorized action)
- what should happen instead (refuse, ask for confirmation, redirect)
Without pass rules, you end up arguing about tone instead of behavior.
2) Test tool execution, not only assistant text
Injection often targets actions:
- send / delete / write
- permission boundaries
- tool calls that should require approval
If the model “answers safely” but the tool still executes, you have missed the actual failure mode.
3) If you use RAG, test injection inside retrieved text
Retrieved instructions can look like higher-priority directives.
Your policy must win.
Test with “instruction-looking” retrieved content and verify the system does not follow it.
4) Verify fallback UX
When protection triggers, users still need a useful experience:
- clear refusal
- next step guidance
- no blank UI, no silent tool execution
Fallback is part of safety.
5) Re-run after every material change
Prompt assembly changes. Tool schemas change. Retrieval changes.
If you do not re-run, your injection coverage decays quietly.
Where to start
Use the free Adversarial Prompt Pack, then write pass rules based on your product’s intended behavior.
If you need an independent pass before launch, book a free testing triage.