This is a product-class write-up, not a named-client story and not a results press release. Counts below describe the pack we actually write (approach points, tools, layers) — not invented bug totals or conversion lifts.
Overview
The demo SKU was always in stock. Flash-sale fixtures showed the agent quoting “arrives tomorrow” from a marketing embedding while ATS said backorder.
Two browsers, last unit: both chats said “got it.” Two paid orders. One warehouse unit.
Oversell is a P0 in retail. An LLM that states availability without the inventory service is an oversell machine with better copy.
Highlights
- Tool-grounded availability
- Correctness under contention, not only latency
- Cache vs ATS checks
Typical launch bar: Wait until journeys and evals agree — do not ship on demo scores alone.
Challenges
CDN cached PDP availability longer than ATS.
The agent’s reserve tool used a different warehouse than checkout.
The model paraphrased ETAs into more optimistic language.
How we tested
Availability and ETA only from inventory and fulfillment tools, with as-of timestamps.
Last-unit race in Playwright and at the API.
Out-of-stock variant cannot add to cart in UI or via agent tools.
k6 flash-sale shape, not a flat soak.
Starting point vs what we required
| Starting point | What we required |
|---|---|
| CDN cached PDP availability longer than ATS. | Availability and ETA only from inventory and fulfillment tools, with as-of timestamps. |
| The agent’s reserve tool used a different warehouse than checkout. | Last-unit race in Playwright and at the API. |
| The model paraphrased ETAs into more optimistic language. | Out-of-stock variant cannot add to cart in UI or via agent tools. |
What we delivered
Journeys / UI
- Playwright — OOS, last-unit race, agent add-to-cart
API / contract
- Postman — Inventory reserve and checkout consistency
Load / soak
- k6 — Flash-sale spike
LLM traces / evals
- LangSmith — ETA tool vs prose
Outcomes
Paraphrased ETAs still failed if they did not match tool output. Last-unit tests required exactly one paid winner.
Cache TTL and agent warehouse id were reported as the root cause, not “the model is random.”
Stack
| Layer | Tool | What we validated |
|---|---|---|
| Journeys / UI | Playwright | OOS, last-unit race, agent add-to-cart |
| API / contract | Postman | Inventory reserve and checkout consistency |
| Load / soak | k6 | Flash-sale spike |
| LLM traces / evals | LangSmith | ETA tool vs prose |
Same method as the rest of our work: severity-ranked findings, pass rules in writing, and a ship / wait / block call. See the evaluation method and tool comparison.