This is a product-class write-up, not a named-client story and not a results press release. Counts below describe the pack we actually write (approach points, tools, layers) — not invented bug totals or conversion lifts.
Overview
The copilot answered “how is my child doing?” for a parent with two linked students — then mixed the children. A student token could retrieve another roster row through the tools API.
Generated worksheets included real student first names from the class list that had been stuffed into the prompt “for personalization.”
FERPA and COPPA shape what you log and what models may train on. District chatbot rollouts have also stalled when vendors and privacy reviews could not keep up. Isolation is the launch gate.
Highlights
- Role-scoped tools, not prompt-only rules
- COPPA / vendor data-use verification
- Peak-path performance with correctness
Typical launch bar: Wait until journeys and evals agree — do not ship on demo scores alone.
Challenges
Personalization and privacy pulled in opposite directions.
Under-13 accounts needed COPPA-constrained vendors; the default LLM route was the consumer API.
School Wi-Fi and a 40-student submit spike were nothing like the office demo.
How we tested
Role matrix: student, parent (linked children only), teacher (this class), admin (this school).
IDOR on gradebook and submissions.
No training on student work by default; check vendor data-use flags.
Peak submit during a class period — lost homework is a P0.
Starting point vs what we required
| Starting point | What we required |
|---|---|
| Personalization and privacy pulled in opposite directions. | Role matrix: student, parent (linked children only), teacher (this class), admin (this school). |
| Under-13 accounts needed COPPA-constrained vendors; the default LLM route was the consumer API. | IDOR on gradebook and submissions. |
| School Wi-Fi and a 40-student submit spike were nothing like the office demo. | No training on student work by default; check vendor data-use flags. |
What we delivered
API / contract
- Postman — Gradebook IDOR and parent linking
Journeys / UI
- Playwright — Parent vs student vs teacher views
Adversarial evals
- Promptfoo — Prompt injection to dump roster
Load / soak
- k6 — Class-period submit spike
Accessibility
- axe DevTools — Public-facing parent login
Outcomes
Tool calls carried the authenticated student/parent scope; extra ids in the prompt were ignored. Names were not injected into generation prompts.
k6 on submit plus Postman gradebook IDOR were the release checks. Consumer-API training flags were treated as a Block until the school-approved endpoint was used.
Stack
| Layer | Tool | What we validated |
|---|---|---|
| API / contract | Postman | Gradebook IDOR and parent linking |
| Journeys / UI | Playwright | Parent vs student vs teacher views |
| Adversarial evals | Promptfoo | Prompt injection to dump roster |
| Load / soak | k6 | Class-period submit spike |
| Accessibility | axe DevTools | Public-facing parent login |
Same method as the rest of our work: severity-ranked findings, pass rules in writing, and a ship / wait / block call. See the evaluation method and tool comparison.