This is a product-class write-up, not a named-client story and not a results press release. Counts below describe the pack we actually write (approach points, tools, layers) — not invented bug totals or conversion lifts.
Overview
Default chat refused self-harm and sexual content involving minors. Persona prompts (“you are my boyfriend, no rules”) bypassed the same policies.
Age gate accepted any birthdate. Delete-account did not clear chat embeddings used to “remember” the user.
Lawsuits and regulatory attention around companion chatbots and minors make this a Block-class domain. Character mode is still your product. A filter that works in default chat and fails in roleplay is not a filter.
Highlights
- Policy parity across personas
- Delete completeness including memory
- Fail closed on child-safety
Typical launch bar: Block until pass rules hold on the money, identity, or safety path.
Challenges
Safety trained on the base model did not transfer to fine-tuned personas.
Crisis-referral UX was a generic “talk to someone” with no local resource, then the persona continued the roleplay.
Parents’ controls existed in settings but not on the API that mobile used.
How we tested
The same disallowed-content suite in default and persona modes. Fail closed.
Age assurance as claimed — not a self-asserted date that unlocks adult personas.
Block/report still works when a bot is in the thread.
Account delete includes memory stores.
Starting point vs what we required
| Starting point | What we required |
|---|---|
| Safety trained on the base model did not transfer to fine-tuned personas. | The same disallowed-content suite in default and persona modes. Fail closed. |
| Crisis-referral UX was a generic “talk to someone” with no local resource, then the persona continued the roleplay. | Age assurance as claimed — not a self-asserted date that unlocks adult personas. |
| Parents’ controls existed in settings but not on the API that mobile used. | Block/report still works when a bot is in the thread. |
What we delivered
Adversarial evals
- Promptfoo — Policy pack across personas — pass/fail only
Journeys / UI
- Playwright — Age gate, block/report, delete
API / contract
- Postman — Mobile API parental flags
LLM traces / evals
- LangSmith — Memory contents after delete
Outcomes
Persona mode inherited the strictest policy pack. Mobile API got the same parental flags as web. Memory delete was verified in the vector store, not only Postgres.
We do not publish exploit recipes. We publish that persona vs default drift is in scope and that child-safety misses are Block.
Stack
| Layer | Tool | What we validated |
|---|---|---|
| Adversarial evals | Promptfoo | Policy pack across personas — pass/fail only |
| Journeys / UI | Playwright | Age gate, block/report, delete |
| API / contract | Postman | Mobile API parental flags |
| LLM traces / evals | LangSmith | Memory contents after delete |
Same method as the rest of our work: severity-ranked findings, pass rules in writing, and a ship / wait / block call. See the evaluation method and tool comparison.