We let agents report their own progress in plain language. The
status feed read like work was happening — "still running," "firing up all 27 engines,"
"saved to memory," "task complete." We believed the narration and moved on.
The words were the most plausible next tokens, not a record of anything
that occurred. An LLM "generates the most contextually plausible next token," and after a
chain of partial actions a confident success message is simply the statistically expected
output — the model has no internal state
tracking whether the file write or the tool call actually executed. So when a step failed
silently, the agent "sees nothing indicating failure. It reports success because success
is the expected outcome." Researchers have a name for this: agentic actions that are
unfaithful to execution history or environment observations,
a failure mode that persists even in frontier models. In our case the engines were not all
firing, and "saved to memory" had saved nothing — but the feed looked productive.
An agent may only use the word done (or acted, saved, sent, running)
when a result actually came back, and it must leave a receipt behind. We added a
proof-of-execution layer: every real action writes one append-only line to an action log
(what, when, the return value), and the honesty clause in the system prompt forbids any
completion claim that is not backed by a returned result. The principle is borrowed
directly from the Proof of Action Protocol — never synthesize a confirmation without real
proof, because while "an LLM can hallucinate 'I saved it,'" it is far harder to hallucinate
a real file path, three specific lines of file content, and an accurate timestamp simultaneously.
A receipt is cheap and it is checkable. "Done" without one is just narration.
charge), require three things back: the exact target (path, URL, record id), a quoted
slice of the actual result (tail the file, read the row, the HTTP status), and a
timestamp. No receipt, no completion claim.
"I acted" only when a tool returned a value. "Started," "kicking off," and "should be
running" are not completion — treat them as in-flight, not done.
witness to whether it worked; the executor and the reporter are the
same blind spot.
Verify against the environment — re-read the file, re-query the count, hit the health
endpoint — before you trust the summary.
cannot rewrite after the fact. A status dashboard should render from the receipt log,
not from the agent's prose.
that as a failure, not a pass. Silence is the bug, not the absence of an error.
Tie this back to the sister rule on this shelf: a blocker isn't real until a command proves
it, and a task isn't done until a receipt proves that. Verify the gate going in; demand the
receipt coming out.