This is a product-class write-up, not a named-client story and not a results press release. Counts below describe the pack we actually write (approach points, tools, layers) — not invented bug totals or conversion lifts.
Overview
The demo stream was live. When the link dropped, the last value stayed on screen and the model said “stable.”
Timestamp source mixed UTC and plant local time in prose.
Industrial assistants fail on freshness and fail-safe, not on eloquence. A missing stale-data indicator is how people act on a dead picture.
Highlights
- Age as a first-class field
- Fail-safe independent of the LLM
- No silent interpolation
Typical launch bar: Block until pass rules hold on the money, identity, or safety path.
Challenges
Product called a 15-minute warehouse rollup “real time.”
The model interpolated missing points.
Mobile operators on bad networks saw even older caches.
How we tested
Every live claim includes age. Past a threshold, the assistant must refuse present-tense status.
Fail-safe UX when the link drops.
Alarm ack/escalate still works without the LLM.
Documented freshness SLO, measured.
Starting point vs what we required
| Starting point | What we required |
|---|---|
| Product called a 15-minute warehouse rollup “real time.” | Every live claim includes age. Past a threshold, the assistant must refuse present-tense status. |
| The model interpolated missing points. | Fail-safe UX when the link drops. |
| Mobile operators on bad networks saw even older caches. | Alarm ack/escalate still works without the LLM. |
What we delivered
Journeys / UI
- Playwright — Stale banner, dropped link, ack without AI
API / contract
- Postman — Tag age headers
LLM traces / evals
- LangSmith — Whether the model saw last-good vs live
Load / soak
- k6 — Many points / high frequency UI lag
Outcomes
We forced a dropped-link fixture. Present-tense answers without a fresh tag failed. Interpolation was labeled estimate or forbidden.
Core alarm path was tested with the copilot disabled — the plant cannot depend on a model for fail-safe.
Stack
| Layer | Tool | What we validated |
|---|---|---|
| Journeys / UI | Playwright | Stale banner, dropped link, ack without AI |
| API / contract | Postman | Tag age headers |
| LLM traces / evals | LangSmith | Whether the model saw last-good vs live |
| Load / soak | k6 | Many points / high frequency UI lag |
Same method as the rest of our work: severity-ranked findings, pass rules in writing, and a ship / wait / block call. See the evaluation method and tool comparison.