This is a product-class write-up, not a named-client story and not a results press release. Counts below describe the pack we actually write (approach points, tools, layers) — not invented bug totals or conversion lifts.
Overview
The copilot summarized spend well on demo accounts. On staging with pending authorizations, multi-currency cards, and overdraft buffers, it quoted a cash figure that was not the ledger balance.
Worse: “You’re good to go” on a credit-limit question read as approval. Support then had to unwind customer expectations.
Customer-facing financial assistants inherit the same liability pattern as policy chatbots: a confident wrong number is an operational and regulatory incident, not a copy issue.
Highlights
- Tool-grounded numbers only
- Forbidden commitment language
- Ledger checks beside LLM evals
Typical launch bar: Block until pass rules hold on the money, identity, or safety path.
Challenges
Pending vs posted vs available were three numbers; the prompt used the word “balance” for all of them.
The model rounded FX in prose and disagreed with the statement PDF.
Sandbox banks delayed webhooks; the copilot filled the gap with a guess instead of “balance not yet updated.”
How we tested
Every money figure must come from a tool call, displayed with as-of time, not from model memory.
Forbidden language list: approved, guaranteed, waived, you can spend. Those require the real decisioning API or a refuse.
Idempotent transfer and IDOR on account ids sit in the same sprint as the evals — UI hide is not authorization.
Support impersonation must be scoped and audited; the copilot must not see another customer’s ledger because a ticket id was pasted.
Starting point vs what we required
| Starting point | What we required |
|---|---|
| Pending vs posted vs available were three numbers; the prompt used the word “balance” for all of them. | Every money figure must come from a tool call, displayed with as-of time, not from model memory. |
| The model rounded FX in prose and disagreed with the statement PDF. | Forbidden language list: approved, guaranteed, waived, you can spend. Those require the real decisioning API or a refuse. |
| Sandbox banks delayed webhooks; the copilot filled the gap with a guess instead of “balance not yet updated.” | Idempotent transfer and IDOR on account ids sit in the same sprint as the evals — UI hide is not authorization. |
What we delivered
LLM traces / evals
- LangSmith — Prove amounts came from the balances API
API / contract
- Postman — IDOR and idempotent transfers
Journeys / UI
- Playwright — Transfer, statement, copilot side-by-side
Adversarial evals
- Promptfoo — Approval-language and social-engineering prompts
Load / soak
- k6 — Payday-window transfer latency without silent queue drop
Outcomes
We required structured tool output for amounts. Free-text numeric claims without a matching tool trace failed the eval.
Postman: same idempotency key twice equals one movement; user A cannot transfer from account B. Those stayed in the launch pack.
Stack
| Layer | Tool | What we validated |
|---|---|---|
| LLM traces / evals | LangSmith | Prove amounts came from the balances API |
| API / contract | Postman | IDOR and idempotent transfers |
| Journeys / UI | Playwright | Transfer, statement, copilot side-by-side |
| Adversarial evals | Promptfoo | Approval-language and social-engineering prompts |
| Load / soak | k6 | Payday-window transfer latency without silent queue drop |
Same method as the rest of our work: severity-ranked findings, pass rules in writing, and a ship / wait / block call. See the evaluation method and tool comparison.