My agents got more reliable when I gave them less to do
Six steps I moved out of the model loop and into plain scripts: what each one cost while a model did it, what changed after, and the three times the scripts broke instead.
On 13 October my fleet spent three runs in a row on one button.
Run 128 tried to publish a queued post and logged publish hung, unverified, no transition. Run 129, two and a half hours later, checked, found the post had never gone out, and logged confirmed unposted, publish button disabled. Run 130 patched the publishing script and logged still disabled after script patch. Each of those was a full model run (a fresh agent reading its instructions, its state files and the playbook, deciding what to do, doing it), and a run like that costs 6 to 8 points of my five-hour subscription window. Three of them went on a button.
The fix, when it came around noon the next day, was a small change to the browser script: a link field kept focus, and the Publish button stayed disabled until something blurred it. A model was the right tool for finding that. It was the wrong tool for noticing that it hadn't found it yet, three times, at full price.
That week I started keeping a list of everything the fleet's agents do that has a right answer a script can compute. Whatever was on the list stopped being the model's job.
The fleet, briefly
The fleet is a set of autopilots on my Mac that run unattended, up to 8 runs a day each, building and launching small digital products. I steer; the agents do the work. I've written before about the first thing it taught me: a rule in a prompt is a request, so the check that matters has to live where the agent can't edit it.
This post is about the steps after that one. The verifier stops an agent from claiming something it didn't do. It does nothing about an agent doing the right thing badly, slowly, or forty times.
The test
Here's the test I settled on, and it's the position in this post you're most welcome to argue with:
If I can write the if-statement, the model shouldn't be the if-statement.
Is the fleet paused? Is there budget left in this window? Has this lane repeated itself three times? Is a post due? Did the post actually go live? Every one of those has an answer that doesn't depend on taste. A model can get them right most of the time. A ten-line function gets them right every time, or gets them wrong the same way every time, which turns out to matter more.
What's left for the model is the part with no if-statement: what to build next, what to write, how to read a page that changed its layout, what a failing log line probably means.
One hourly batch. Five of the six boxes are plain scripts. The blue one is the only place a model gets a say, and it's also the only one I'd struggle to unit-test.
1. Checking claims (September)
This one came first, and I covered it in the earlier post, so I'll only say how it fits the test. "Does this record that says listed have evidence of a live listing?" is an if-statement. Asking the agent that wrote the record whether it was telling the truth was never going to work. A 126-line script that runs after every run and demotes anything unbacked does.
2. Pausing, with an end date (22 September)
I asked for a two-day stop while I finished some setup. The fleet's pause was a file, state/paused: if it exists, nothing runs. That's fine until the human who said "two days" forgets, and then it's a two-week stop.
So the pause file can now hold a timestamp, and the guard lifts the pause itself when the time passes:
case "$deadline" in
"") echo "paused (state/paused exists)"; return 0 ;;
[0-9][0-9][0-9][0-9]-[0-9][0-9]-[0-9][0-9]T*Z)
if [ "$deadline" \> "$(date -u +%FT%TZ)" ]; then echo "paused until $deadline (state/paused)"; return 0; fi
rm -f state/paused; return 1 ;;
# Anything else is a note, not a deadline. A note never lifts a pause.
*) echo "paused ($(head -c 60 state/paused | tr -d '\n'))"; return 0 ;;
esac
From the fleet's preflight guards. The last branch is my favourite line in the file: if I write a note in the pause file instead of a date, the fleet stays paused, because a typo should never be what un-pauses it.
No agent is ever asked whether it's time to resume. It never sees the question.
3. Budget: reading the real number (22–26 September)
The fleet runs on a Claude subscription I also use myself. The first version estimated usage from local logs and gave the autopilots a share of it. On 22 September I replaced the estimate with a read of the actual subscription windows (the same numbers Claude Code's /usage shows), recorded before and after every run.
Four days of readings showed me the estimate was fiction. A single run cost 6 to 8 points of the five-hour window and 0.5 to 1 point of the weekly one, which meant the autopilot alone would have emptied a week in three days. The share I'd assumed for the autopilot was borrowed from another setup and was simply wrong here. I'd have kept believing it if a model had been doing the pacing, because a model doing the pacing would have been reading the same estimate.
Now the gate is a threshold on a measured number, and it says why it skipped. From the run log on 21 October:
SKIP portfolios/[lane] — limits: seven_day at 95% is 5.0 points over its 90% mark; resets 2026-10-25T07:59:59Z
The weekly window was nearly gone. The gate kept the fleet from spending the rest, and the same line says when the window resets.
4. Finding the binary (26 September), or: the script was the bug
Here's the first time moving a step into code bit me.
The runner had the path to the claude binary hardcoded to where Homebrew used to install it. On 23 September the Homebrew cask was replaced by the native installer, which lives somewhere else. From then on, every run died about five seconds in, and the runner logged it as tick log unparseable. Over 23–26 September that line appears 90 times: 9, then 22, then 47, then 12 before the fix. Three days of total failure, and nothing about it was loud.
The fix was a function that looks in the places the binary can be, picks the first one that's executable, and fails the batch if none are:
resolve_claude() {
local c
for c in "$HOME/.local/bin/claude" /opt/homebrew/bin/claude "$(command -v claude 2>/dev/null)"; do
[ -n "$c" ] && [ -x "$c" ] && { echo "$c"; return 0; }
done
return 1
}
This is the honest cost of the whole approach. Deterministic code fails too, every single time, at machine speed. What it doesn't do is fail differently every time. Ninety identical log lines are a bug report. Ninety different, plausible-sounding agent summaries would have been a mystery.
5. Deciding it's stuck (14 October)
Back to the button. The real problem on 13 October wasn't that the agent couldn't fix the button: it eventually did. It was that each run started fresh, saw the same unfinished item, and tried again, and nothing in the system thought three identical failures were worth mentioning.
The stall rule is a script that runs before each run and reads git. A run counts as stuck if its commits touched none of the files that represent progress (the publish queue, the products, the ledger). Three stuck runs in a row on the same work item, and the lane is skipped and one blocker is filed for me. Why three: each autopilot gets at most 8 runs a day, so three is 3 to 9 hours of retries, and every transient failure in the history cleared on the very next run. Two would also have parked a lane that was deliberately killing a product.
It shipped at 12:13 on 14 October. That night it parked a lane after runs 134–136, which is exactly what it was for.
And then it went quiet for four days.
Runs 134–136 were stuck because the lane had used its daily publishing cap: nothing to do, which looks identical to "can't do anything" if all you count is files touched. The park had no expiry, so from 15 to 18 October that lane ran zero times and published nothing, and the only signal was an absence.
The fix on 19 October did two things. Runs that stopped because of a cap no longer count as stuck. And a park lasts one UTC day: the next day one run goes ahead, and if it repeats the same outcome, the lane stays parked and a blocker goes onto my dashboard. A park can't go silent for days again, because silence is now a state the script reports.
6. Publishing (19 October)
This was the big one, and the reason I started the list.
Before 19 October, a worker run published posts. That meant the agent loaded its whole context, decided a post was due, drove the browser, and then reported whether it had worked. Every published post cost one model run, and every "published" was the agent's own account of what happened.
Now there's a pacer, a Python script that runs at the start of every hourly batch, before any model is involved. Its rule is short. A post is due if there's an unposted row in the queue, today's count is under the cap (6 a day), and the last post is at least 120 minutes old. If it's due, it publishes exactly one, then reads the account's own list of posts back and records the URL it finds there (never the URL the browser happened to be on). The publishing script refuses to run unless the pacer is the one calling it. Worker runs now only refill the queue.
I grepped both scripts for any call to a model before writing that paragraph: there isn't one.
Here's the part where I'd love to show you a hockey stick, and can't. Posts per day from the queue file:
before (a model run per post) 10-10: 3 10-11: 4 10-12: 2 10-13: 1 10-14: 4
parked, see above 10-15 .. 10-18: 0
after (the pacer, no model) 10-19: 2 10-20: 1 10-21: 2 10-22: 6 10-23: 5
10-24: 4 10-25: 0 10-26: 2
14 posts in five days before, 22 in eight days after. The averages are about the same.
The low days after the switch each have a cause I can name. 19 October was a partial first day. On 20 and 21 October, the platform's list sometimes took a while to show a new post, the pacer treated "not in the list yet" as a failure, and stopped for the day. The 22 October fix holds a clean publish as pending and re-reads it on later runs, and that day was the first time the pacer reached its cap: 6 of 6. On 25 October nothing ran at all, because the home-network guard couldn't read the router's address and skipped 21 batches that day. It's built to fail closed (the fleet doesn't run on networks it can't identify), so a day of nothing is the guard doing its job, if annoyingly.
So the pacer didn't make posting faster. What it did:
- Posting went from one model run per post to zero, so those subscription points go to work only a model can do.
- Every recorded URL was read off the platform after the fact, by code, so "published" means something I can check.
- When it fails, it fails with a named reason, and I can fix that reason once instead of watching three runs rediscover it.
The one from 27 October
I wasn't planning to have a fresh example, but the stall rule supplied one early on 27 October, while I was drafting this. It parked a lane after runs 172–174. Each of those runs said it had added a new section to a reference page the lane maintains: sections 22, 23 and 24. I checked the live page while writing this, and all three sections are there.
So why parked? Because that page lives in a different repository, and the stall rule only counts commits to the files it watches. In the fleet's repo, those three runs touched a journal and a metrics file and nothing else, which by the rule's definition is no progress.
I didn't clear it, and I didn't have to. Since the 19 October fix a park lasts one UTC day, so the lane got one run later that day. Run 175, at 16:53, turned sections 22–24 into three posts in the publish queue. The queue is a file the rule watches, so that counted as progress and the lane carried on.
Then it did the same thing four more times. The rule parked the lane once on each UTC day from 28 to 31 October (runs 182–184, 189–191, 193–195 and 197–199), and every cycle had the same shape: a few runs that added sections, audited listings or read metrics without touching a watched file, then a park. The first three parks were each lifted by the next day's single run refilling the queue with three posts, which counts as progress and lets the lane run again. The fourth was still parked when I published this. The page is up to section 40 now (I checked that one too). Sales are still zero.
Two things in that loop are worth pulling apart. The first is the one I wanted: the pacer published through four of the five parks, 18 posts from 27 to 31 October, four of them on the 31st after that day's park. Publishing no longer waits for the worker to be allowed to run. (During the fifth, on 30 October, the pacer was waiting out a failed publish from that morning, which it retries the next UTC day, and then the home-network guard skipped every batch from mid-morning to evening. Both of those are scripts saying no for a reason they wrote down.)
The second is a gap I didn't see coming. The stall rule escalates only when the resume run repeats the stuck outcome, and a queue refill never does, so the rule that exists to ask for me never asked. What did flag it was a blunter check on my dashboard. It counts runs in a row on the same work item with no publish or stage change, a queue refill doesn't reset it, and it has listed this lane as needing me since the streak started on 26 October. This morning it read 28 runs in a row on one item. Two scripts were watching the same lane, and the one with less nuance was the one that got it right.
So the stall rule was right that something was off, and wrong about what. The sections were real work that reached the watched files a run late, and the rule can't see across repositories. Whether forty reference sections are the best use of a lane with no sales is a judgement call with no if-statement in it, which by this post's own test makes it the model's question, or mine. Getting that question in front of me is a script's job, and the stall rule should be doing it too. The fix I'd make is to have it count parks per week as well as repeats per resume. It isn't built yet.
What's still the model's job
I'm not working toward a fleet with no model in it. Everything above is plumbing, and the plumbing used to sit inside the agent's context, competing with the actual work. Now a worker run starts already knowing it's allowed to run, that there's budget, that it isn't stuck, and that publishing is someone else's problem. What it's asked is the hard part: what should this lane build next, and is the thing it built any good?
That's also why I'd call it more reliable and not just cheaper. The failures I see now are mostly of two kinds: a script that's wrong the same way every time (the binary path, the four silent days, the slow list), which I fix once; and a model making a judgement call I disagree with, which is the conversation I actually want to be having with it.
What I'd tell you to do
- Go through one day of your agent's logs and mark every decision that had a right answer. Each one is a candidate for a script.
- Start with the questions that run before the agent does: should it run, is there budget, is it stuck. They're the cheapest to move and the agent shouldn't be the one answering them anyway.
- Read the real number. If your budget, quota or rate limit comes from an estimate, the agent and the guard are both pacing against the same wrong figure.
- Make every guard print why it said no. A skip with a reason is a log line. A skip without one is four quiet days.
- Give every "stop" an expiry and an escalation, and count stops across days, not only in a row. A park that lasts forever is an outage nobody gets paged for, and a park that comes back every day is the same outage in instalments.
- Expect the scripts to break, and be glad they break the same way every time.
Thanks for reading. Next up: the same idea pointed at my own coding agent, where the rules I care about most ("never push to a work repo from a personal one") moved out of the prompt and into a hook that denies the tool call. It's public, with its tests: agent-setup. Follow on RSS or X @denzilnelsonai. If you've moved a step out of your agent's hands and it got worse, I'd especially like to hear about it.