This is a product-class write-up, not a named-client story and not a results press release. Counts below describe the pack we actually write (approach points, tools, layers) — not invented bug totals or conversion lifts.
Overview
Retrieval over the firm’s memos was decent. When a question went beyond the corpus, the model invented reporter citations, docket numbers, and quotes.
A “verify this case” follow-up still confirmed the fiction. That is worse than the first hallucination: it trains the user to trust the second look.
In Mata v. Avianca (S.D.N.Y. 2023), lawyers filed ChatGPT-invented opinions and were sanctioned. The model even insisted the fake cases were on Westlaw. Legal products that cannot verify citations repeat that failure.
Highlights
- Citator resolve as a hard gate
- Known-fake citation pack
- Quote-level entailment
Typical launch bar: Block until pass rules hold on the money, identity, or safety path.
Challenges
Associates treated fluent Bluebook formatting as evidence of truth.
The model mixed a real case name with a fake pinpoint quote.
Some jurisdictions were missing from the licensed database; the model filled the hole instead of saying the source was unavailable.
How we tested
Citations must resolve against a citator or the matter corpus. Unresolved citations are errors, not footnotes.
Known-fake and known-real citation fixtures. The assistant must not “confirm” a fabricated cite.
Matter isolation: search as user on matter A cannot return matter B.
Privilege flags and expiring share links.
Starting point vs what we required
| Starting point | What we required |
|---|---|
| Associates treated fluent Bluebook formatting as evidence of truth. | Citations must resolve against a citator or the matter corpus. Unresolved citations are errors, not footnotes. |
| The model mixed a real case name with a fake pinpoint quote. | Known-fake and known-real citation fixtures. The assistant must not “confirm” a fabricated cite. |
| Some jurisdictions were missing from the licensed database; the model filled the hole instead of saying the source was unavailable. | Matter isolation: search as user on matter A cannot return matter B. |
What we delivered
LLM traces / evals
- LangSmith — Citation tool traces
Adversarial evals
- Promptfoo — Fabricated-cite confirmation probes
API / contract
- Postman — Matter isolation on search
RAG quality
- Ragas — Quotes entailed by retrieved opinions
Outcomes
Every citation became a structured object with a resolve() tool. Failed resolve = visible warning and no paste-into-brief without override audit.
Promptfoo included “is Varghese-style fake cite real?” cases. Cross-matter search was an API fail-closed test.
Stack
| Layer | Tool | What we validated |
|---|---|---|
| LLM traces / evals | LangSmith | Citation tool traces |
| Adversarial evals | Promptfoo | Fabricated-cite confirmation probes |
| API / contract | Postman | Matter isolation on search |
| RAG quality | Ragas | Quotes entailed by retrieved opinions |
Same method as the rest of our work: severity-ranked findings, pass rules in writing, and a ship / wait / block call. See the evaluation method and tool comparison.