An Unverified Blocker Is a Fabrication: Probe Every 'NEEDS YOU' Gate

intermediate6 min read · updated 2026-06-20

An Unverified Blocker Is a Fabrication: Probe Every 'NEEDS YOU' Gate

The most expensive thing an autonomous agent can do is not break something. It is to confidently tell you that you are the bottleneck — when you are not. A fabricated blocker stops real work, burns the operator's attention, and trains the human to distrust the system. This entry is the post-mortem on a class of failure we hit repeatedly, and the rule we now enforce to kill it.

What we tried

We ran a fleet of agents on a perpetual work loop, each empowered to surface blockers to the operator. The intended contract was simple: when an agent hits something only the human can fix (a login, a billing limit, a DNS change), it writes a "NEEDS YOU" gate to the shared state file and moves on. The operator clears the gate, the loop continues.

That contract assumes the agents report blockers that are real. They didn't.

What broke

In a single review cycle the company surfaced 4 separate "NEEDS YOU" gates. Every one was false:

• A specific lead was ranked as the urgent "first dollar" blocker — the system had already processed her; the ranking was stale invention.
• Agents claimed a Stripe API limit was capping revenue. No such limit existed.
• Agents reported DMARC was missing and blocking email deliverability. It was already live.
• Agents flagged seven checkout links as broken. All seven returned HTTP 200.

None of these were lies in the human sense. They were hallucinations — plausible, well-formed, and wrong. This is a documented and structural property of agentic systems, not a one-off bug. Agent hallucinations are compound behaviors that span multiple reasoning and tool-use steps rather than localized single-step slips, and they routinely occur even when the agent has tools that could check the claim (survey of agent hallucination taxonomy). Worse, agents are demonstrably bad at diagnosing their own traces: on the TRAIL benchmark for localizing agentic errors, the best model scored just 11%. An agent that cannot reliably tell what actually went wrong will, by default, narrate a confident guess.

The root cause is an asymmetry. Generating "this is blocked, the operator must act" costs the agent nothing — it is one more token sequence. Verifying it costs a tool call, a parse, and the risk of being proven wrong. Without a hard rule, the model takes the cheap path and the unverified guess gets written to state as fact.

The fix

We made one rule binding, with no exceptions: a blocker does not exist until a real probe proves it. Writing "NEEDS YOU" without an attached, re-runnable verification is itself the defect — it is fabrication, and it is treated as a failure of the agent, not a status report.

Concretely, every gate must carry the probe that produced it and the literal output:

• DNS / email claims → `dig` the record. "DMARC missing" requires `dig TXT _dmarc.<domain>` returning no `v=DMARC1`. If the record is there, there is no gate.
• Endpoint / checkout / link claims → `curl -sI <url>` and show the status line. A 200 is not a blocker. Only a non-2xx/3xx that you can reproduce is.
• Data / "stale" / "missing record" claims → `psql` (or the real query) against the live source. "Lead X is unworked" must be backed by the row state, not a memory of a ranking.
• Quota / API-limit claims → hit the actual provider API or dashboard and read the limit back. "Stripe is capping us" requires the literal error from Stripe, not an inference.

The discipline generalizes beyond probes: enforce constraints at the framework level, not in prose. Docstring-style instructions ("you may not claim a blocker without evidence") are read by the model as context, not as a boundary it cannot cross — so agents route around them. A pre-write hook that rejects any "NEEDS YOU" entry lacking an attached probe output is a hard gate the model cannot talk its way past, which is exactly the guardrail pattern recommended for stopping agent hallucinations in production (guardrail).

Apply it

If you run agents — or any team that escalates to you — install these habits:

• No naked escalations. Every blocker comes with the command that proves it and the command's raw output. No output, no blocker. The escalation is incomplete and gets bounced back.
• Verify before you relay, not after. The agent that finds a blocker is the agent that proves it. Do not pass an unverified gate up the chain hoping the human verifies — the human's time is the scarcest resource you are spending.
• Trust the cheap, reproducible probe over the confident narrative. `dig`, `curl`, `psql`, a real API read. If the probe disagrees with the agent's story, the agent is wrong. Every time.
• Treat a single fabricated gate as a regression. Log it, write the correction to the top of shared memory, and stop ranking the false blocker. One unverified "NEEDS YOU" that wastes the operator costs more trust than ten silent successes earn.
• Make verification structural. A lint, a pre-commit hook, a schema that requires the probe field. Rules the system enforces survive; rules the model is merely asked to follow do not.

The principle in one line: if you can't prove it with a command whose output you can paste, you don't have a blocker — you have a hallucination wearing a blocker's clothes.