- Use three outcomes: Ship, Wait, Block.
- Base the call on coverage you can point to, not demo energy.
- Block when data exposure, ungated write actions, or missing fallbacks are unknown on user-facing paths.
- Wait when coverage is partial but blast radius can stay limited.
- Ship only when critical probes are done and open issues are severity-ranked with owners.
- A named person must own residual risk. A group shrug does not count.
Why AI launches need an explicit gate
AI features fail in ways that look fine in a happy-path review:
- Confident wrong answers on primary jobs
- Tool calls that should never have fired
- Context leaking across roles or tenants
- Timeouts that strand the user
- Cost spikes from silent retry loops
A binary “LGTM” from a walkthrough does not capture that. A small set of honest questions does. The goal is not bureaucracy. The goal is a decision someone can defend a week later.
The three outcomes
Ship
Use when:
- Critical probes below are in the stronger column, or residual risk is explicitly accepted in writing
- Open findings are severity-ranked with locations and owners
- You know how you will retest after the next prompt or model bump
- Rollout audience matches the confidence you actually have
Ship can still include Medium items. It should not quietly include Critical data or write-path issues.
Wait
Use when:
- Coverage is partial, but rollout can stay narrow
- Open issues have owners and a short time box
- You can limit audience (internal, beta flag, single workspace, read-only mode)
Wait is active. It needs a date and an owner. “Wait forever” is a soft Block that never resolves.
Block
Use when any of these are true and unaccepted:
- Data exposure unknown on a product that sees private data
- Write actions can fire with little friction
- No designed fallback on a core user-facing flow
- The team cannot describe what was tested beyond the demo
Block is not a moral judgment. It is a blast-radius judgment.
The criteria we use in plain language
These match the spirit of our interactive helper. Adjust wording to your product, not the other way around.
1. Adversarial and ambiguous prompts
- Stronger: Documented set, run on purpose, results noted
- Weaker: A few informal tries
- Gap: Not really tested
2. Private context and data exposure
- Stronger: Probed; boundaries held
- Weaker: Partially checked / unsure
- Gap: Not checked
3. Failures and fallbacks
- Stronger: Clear fallback or human handoff
- Weaker: Something happens, not designed
- Gap: Bad or blank state
4. Write / send / delete actions
- Stronger: Approval gate, or no write actions
- Weaker: Soft confirmation only
- Gap: Model can act with little friction
5. Add your product-specific fifth
Examples:
- Retrieval citation honesty for RAG assistants
- Cost circuit breakers for chatty agents
- Admin-only tools exposed to end users
- Payment or refund paths influenced by model output
Write the fifth criterion down so the gate is not vibes.
Worked examples (generic patterns)
These are patterns, not client stories.
Example A: Internal Q&A over company docs
- Adversarial: weaker
- Data exposure: gap (multi-team docs, unclear ACLs)
- Fallbacks: stronger
- Writes: none
Call: Wait or Block on broad access until ACL / retrieval isolation is probed. Internal-only with trusted users may be Wait with a time box.
Example B: Support copilot that drafts replies
- Adversarial: stronger
- Data exposure: stronger for tenant boundaries
- Fallbacks: weaker (blank composer on timeout)
- Writes: send requires human click
Call: Wait until fallback is designed. Ship may be fine after that if send stays human-gated.
Example C: Agent that can create tickets and update fields
- Adversarial: weaker
- Data exposure: unsure
- Fallbacks: gap
- Writes: model can update with soft confirm
Call: Block for external users until write gates and failure states are real. Demo energy does not change this.
Example D: AI feature behind a beta flag for 50 internal users
- Adversarial: stronger for the core job
- Data exposure: stronger inside a single trusted workspace
- Fallbacks: weaker
- Writes: human-gated
Call: Wait is often honest here: keep the flag, fix fallbacks this week, do not widen to external customers until fallbacks and a retest pack exist. Internal beta is not a free pass if writes or private data are involved; it is a smaller audience with the same failure modes.
Documenting the decision so it survives the week
A useful release note is short:
- Date and product surface
- Ship / Wait / Block
- Audience (percent, flag, workspace)
- Top open Critical/High with owners
- What would flip the call (for example: “Block lifts when write gate ships and fallback retested”)
Store it where launch conversations already happen. If the only record is a meeting memory, you will re-litigate every day.
Who owns the final call
Someone with authority to delay launch. QA and testing partners can advise. Calendar pressure should not own severity.
Good practice:
- One release owner
- One written note: Ship / Wait / Block + top open Critical/High + owners
- Same note in the launch channel, not only in a private chat
If sales and eng disagree, the release owner decides with evidence, not volume.
After a Wait: how to avoid limbo
Wait fails when it becomes endless. Protect it:
- Write the exit criteria on day one (“Wait lifts when X and Y are retested”)
- Keep the audience limit enforced in product, not only in a doc
- Review open Critical/High twice a week until the gate flips
- If exit criteria slip more than once, consider Block for the wider audience
Wait is a controlled experiment. Limbo is hope without controls.
After a Block: how to move again
Block is not permanent. Make it temporary by design:
- List the smallest set of probes or fixes that lift it
- Prefer removing write power or narrowing data access over “try harder on tone”
- Retest with a second person before flipping to Wait or Ship
The fastest path out of Block is usually reducing blast radius, then proving the new boundary.
Free tool
Run the same gate language interactively:
Launch go / no-go helper (free, no signup)
Use it in a launch channel thread so the decision is visible. Pair with the launch checklist so Wait items have owners.
How GenCodeQA uses this in engagements
We do not replace your release owner. We give evidence for their call:
- Severity-ranked findings
- Clear location and reproduction
- Retest of fixes when scoped
If the honest gate says Block and the calendar says Ship, the calendar is the problem. We will say that plainly. We will not invent fake urgency, and we will not soften Critical findings to protect a date.
Common false confidence
- Shipping because sales has a date and the demo is pretty
- “Wait” with no owner and no date
- Blocking forever instead of narrowing rollout while you fix
- Treating the helper output as legal or compliance approval
- Accepting residual risk as “the team” with no name
- Re-running the helper until the recommendation feels nicer
When to get a second pair of eyes
If your team disagrees on Ship vs Wait, an independent pass often settles the argument faster than another internal demo.
Bring: product type, model setup, what you have tested, and the go / no-go answers you already disagree about.
FAQ
Who should own the final call?
Someone with authority to delay launch. Advisors can recommend. They should not be the only signature on residual risk.
Can we ship with known Medium findings?
Sometimes, with owners and monitoring. Critical / High on data and write paths usually means Wait or Block.
Does go / no-go replace a checklist?
No. Checklist organizes work. Go / no-go decides. Use both.
What if we only ship to 5% of users?
That can turn a Block into a Wait if the 5% is intentional, monitored, and the open Critical issues are understood. Percentage alone is not a gate.
Should we document accepted risk?
Yes. Name, date, what was accepted, what would reopen a Block. Memory is a bad compliance system.