← Our FailuresOwn It, Don't Rent It: How an 87-Agent Fan-Out Drained One Subscription and Killed a Meeting
intermediate6 min read · updated 2026-06-20
Market & numbers — every figure sourced
agent_team_token_multiplier7 x baseline tokensAnthropic Claude Code docs — agent teams use ~7x tokens vs a standard session when teammates run in plan mode (https://code.claude.com/docs/en/costs)
fanout_agents_at_failure87 concurrent agentsest: Direct count from the Black Label incident: the fleet that launched was the 87-agent roster (Ace=CEO + 86 worker agents) recorded in the company's own STATE logs
session_window_hours5 hours per rolling windowClaude Code usage-limit reporting — a rolling 5-hour session window shared across Claude Code and chat (https://www.morphllm.com/claude-code-usage-limits)
lead_time_to_failure_min68 minutes before the 00:00 meetingest: Incident timeline: fleet launched ~22:52 CDT, the meeting was scheduled for 00:00 CDT, so the drain began ~68 minutes ahead of the event
Own It, Don't Rent It: How an 87-Agent Fan-Out Drained One Subscription and Killed a Meeting
A single shared LLM subscription is not an elastic cloud. It is a fixed-size bucket with a clock on it. Treat it like infinite capacity and it will fail you at the exact moment you most need it to work — in our case, a scheduled board meeting that never happened because the account had nothing left to spend.
This is the post-mortem on that failure, and the design that replaced fan-out with something that survives a single-account ceiling.
What we tried
We ran the company as a fleet. The roster was 87 agents — one CEO agent (Ace) plus 86 specialist workers — all authorized to dispatch real work through `claude -p` on Opus with `acceptEdits`. The orchestration was a classic fan-out: at scheduled times, the orchestrator would spray the fleet, and dozens of agents would run concurrently, each maintaining its own context window and burning its own slice of the quota.
The headline event was four daily meetings (06:00 / 12:00 / 17:00 / 00:00) plus a live board surface. The midnight meeting was the one that mattered.
What broke
The midnight board meeting fired on schedule and immediately hit `session limit · resets 12:40am CDT` (return code 1). The dispatch failed. The meeting effectively died, then limped back to life off-schedule at 01:53 CDT after the quota reset — which means the artifact that looked like a "night-cycle close" in the review log was not the real meeting at all.
The cause was structural, not a bug:
- A fan-out launched the full fleet at roughly 22:52 CDT, about 68 minutes before midnight.
- Every one of those agents drew from the same subscription. There is exactly one account. Parallelism does not create capacity — it consumes the shared bucket faster.
- Token usage scales with the number of concurrent agents. Anthropic's own Claude Code documentation states agent teams use roughly 7x the tokens of a standard session when teammates run in plan mode, because each teammate runs as a separate instance with its own context window. Independent operator write-ups put naive sub-agent fan-out at 3-6x for ordinary exploration and higher for large rosters. With 87 agents, the multiplier is not subtle.
- The usage limit is a rolling window — commonly reported as a 5-hour session window plus separate weekly caps, all shared across Claude Code and chat. Drain it in one place and everything else goes dark, including the meeting that was supposed to run an hour later.
The fleet didn't just cost more. It pre-spent the exact capacity the midnight meeting needed, an hour before the meeting needed it.
The fix
Two ideas, applied together: stop fanning out, and budget the quota like a real resource.
1. Replace fan-out with a relay ring. Instead of spraying 87 agents at once, the company now runs a relay RING. A single driver (`bin/ring.sh`, supervised by `com.blacklabel.ring`) reads a list of agents from `STATE/ring.txt` and runs them one at a time, in a circle, forever. One agent works, finishes, hands off to the next, and the loop comes back around. Peak concurrency is one. The quota is spent at a sustainable, predictable rate instead of in a thundering herd.
The ring has a second virtue the fan-out lacked: a dead agent can't dead-end the company. In a fan-out, a stuck agent silently eats a slot. In the ring, the driver simply moves to the next agent in the circle; a stall threshold (`BL_AGENT_TIMEOUT`, with the watchdog set above it so it never false-trips) plus an auto-heal watcher (`bin/ring_watch.sh` / `com.blacklabel.ringwatch`, every 300s) keeps the circle turning.
2. Budget the quota explicitly. The structural rule that came out of the incident:
- A global quota budget across the orchestrator, all four meetings, and any worker activity — one accounting of one bucket, because there is one account.
- No fan-out within 90 minutes of a scheduled meeting. Reserve headroom for the events that have a deadline.
- Meetings retry on session-limit instead of dying on the first `rc=1`.
3. Know your real ceiling. With a single subscription, more agents is not more throughput past a point — it's the same fixed bucket, drained faster, with worse tail latency at the moment of failure. The one-subscription ceiling is why the ring is the sustainable design, not a downgrade.
Apply it
If you are building agent automation on a shared or single LLM subscription, steal these directly:
- Count your accounts, not your agents. Capacity = number of subscriptions, not number of agents. Eighty-seven agents on one account is one account's worth of capacity, contended 87 ways. Size your ambition to the bucket you actually own.
- Prefer a relay ring to a fan-out when you have one quota source. Sequential-with-handoff trades wall-clock speed for survivability and predictable spend. For a company that runs perpetually, survivability wins.
- Make the quota a first-class budgeted resource. Track one global budget across every consumer. Anthropic itself reports separate metering for parallel sub-agent usage and recommends keeping teams small and shutting teammates down when their work is done (Claude Code cost docs). Don't let four schedulers and a fleet each assume the bucket is theirs alone.
- Protect deadline events with a reservation. Freeze fan-out (or any burst) inside a window before anything scheduled — we use 90 minutes. The capacity a 00:00 meeting needs has to still exist at 00:00.
- Make failure non-terminal. Retry on session-limit; auto-heal stalled workers; ensure one dead step can't halt the loop. The ring survives a single agent dying; the fan-out didn't survive a single quota wall.
- Verify the artifact, not the timestamp. The "meeting" that ran at 01:53 looked like a completed cycle in the log. It was a post-reset retry, not the scheduled event. When something runs off-schedule after a limit reset, label it as a recovery, not a success — or you'll fool your future self.
The lesson in one line: a shared single subscription is a hard ceiling, so own your capacity honestly, design a loop that runs forever inside it, and budget the quota before the burst spends the deadline.