USD 1.61. That is what Pioneer AI & Data logged in model spend over two months for two scheduled agents classifying email, across 508 runs. Six of those runs did not record cost, so the real figure is only slightly higher. Over the same period, their total inventory of agents and services to keep running grew from 13 to 50. The author notes he did not track his own hours closely enough to publish a number.
Another agency, Oase Creative, tracked 134 days of scheduled runs of their agent system. They recorded some failure on 72 of 133 evaluable days, or 54%. That is an indicator of operational load, even though it does not directly measure interventions or human hours. They sum it up themselves: the bill arrives in attention, not in euros.
In my workflow, attention is the cost I care most about cutting: every time I have to watch, review, correct, or redirect an agent.
Those cases describe operational maintenance. Here I focus on another drain on attention: steering agents during development and correcting scope drift. I use three terms consistently: steering is redirecting the agent while it works; drift is the gap between what it implements and what I wanted; outcome is the verifiable result I am after, rather than the solution I have in mind.
Governance vs. autonomy
On one hand we want governance. We want to stay in control, so we add guardrails, reviews, hooks. On the other hand we want autonomy: having the agent finish the job without supervision.
At first those two goals look contradictory. Not every control costs the same, though. When every check ends in a human approval, interruptions pile up. Automated checks, like tests, acceptance criteria, or a Definition of Done (DoD) verified by another agent, can reduce them, although they need maintenance too.
I ran into this on my first attempt at building my own orchestrator harness. I wanted an autonomous yet safe flow, so I put guardrails in place:
- Block any task where the agent was unsure.
- Block tasks that needed credentials.
- Ask for confirmation before deploying to production.
- Classify my request to decide whether it could run directly or needed a plan or design.
During implementation I had to step in and steer: the agent expanded scope with things I had not considered, without checking with me. Once I started using it, inconsistencies surfaced between agents. One agent approved a task for execution, and the implementing agent decided it was not ready, blocked it, and sent it back to review. I was spending more time fixing my orchestrator harness than shipping code.
Adding another reviewer did not fix that discrepancy. The issue was that each agent applied its own criteria. What I needed was a shared rule: the same acceptance criteria decide when a task runs, when it stops, and when it escalates. If the agents still cannot agree, I decide the best approach based on my experience.
Why steering is expensive
Steering usually arrives late. The agent has already built the feature and there is drift from the original idea. Now it has to be corrected, and the agent does not only need to grasp the new context: it also has to understand what is already implemented, figure out how to fix it safely, and iterate further. Every round asks for more attention.
Planning agents or planning modes are the usual way to keep implementation aligned with intent. Agents can hallucinate, though, and reviewing a plan can demand heavy attention before you can run it safely and avoid drift.
And it does not stop there. Once the plan is implemented, you have to verify that the result lines up with it. To do that, I first need to understand the plan and the specs that were generated.
⚠️ Gotcha: a long plan does not eliminate attention, it moves it. If I do not understand the plan, I cannot validate the implementation against it.
A concrete example: I needed a simple interface to edit a system’s markdown context files. What I had in mind was a separate app to edit the .md files. It turned into a migration of the system to embed an editor inside it. I had not been explicit that the tool had to be standalone, nor had I checked with the agent that we were on the same page. The plan resolved that ambiguity on its own.
How I make sure the plan is well defined before running it
The real question is how we make sure plans are well defined and aligned with the result we want to build. These are the strategies I am using and testing before and during plan creation.
1. Adversarial review, ideally with another provider and model
Before a single line of code is written, I hand the plan (not the code) to another agent with an explicit mandate to find holes in it. I test with another model, often from a different provider, to look for different blind spots. In my experience, that helps catch gaps the original agent missed, though switching providers does not guarantee a better review: I evaluate what extra errors it finds and how it justifies them.
For example, you can use an OpenAI model to generate the plan, Claude Opus to review it, and Grok to implement it, or whatever combination works best for your workflow.
For this I use agent-fabric, a CLI that installs the same portable agents across OpenCode, Kilo, Codex, Claude Code, and other tools, with each provider’s model configured per adapter, so I can assign a different provider to the planner role and the reviewer role.
2. Simple plans, scoped to a valuable outcome
In my experience, plans that lock onto a solution too early tend to grow. Short plans with a verifiable outcome and explicit scope boundaries work better for me. With AI we iterate much faster, so it pays to plan lightly and iterate on the outcome.
The common mistake is planning from the solution we picture in our heads instead of from the outcome. The agent finds gaps and hypothetical scenarios, adds features we never asked for, and we end up with huge plans that eat up review time.
3. Use cases translated into test cases
Test cases turn part of the spec into verifiable conditions. Another agent can check them and cut down my routine review; you still have to verify that they actually represent the outcome and its constraints.
For instance, in an authentication system, a use case might be: an anonymous user visits the site and must authenticate before using a service. From that come test cases like these:
- Given a request without a valid session, when it calls the protected service directly, then it is rejected and returns no protected data.
- Given an email with an invalid format, when I request a code, then the system rejects the request and sends no OTP code.
- Given a valid email for a registered user, when I request a code, then the system sends an OTP code to that email.
- Given a valid email that is not registered, when I request a code, then no code is sent, no session is created, and the public response is the same as for a registered user: “If the email is registered, you will receive a code.” Both cases return the same HTTP status, with no exploitable timing differences, so accounts cannot be enumerated. Sending the code is verified internally.
- Given a valid, unused code issued for that account, when I enter it, then a session is created for that account and the code is marked as consumed.
- Given an incorrect, expired, or already used code, when I enter it, then no session is created and the response is “Invalid or expired code.”
- Given the attempt limit has been exceeded, when I enter another code, then it is rejected even if it is correct.
- Given a valid session created after verifying the code, when I call the protected service, then I can use the service I am authorized for.
This example is not a complete authentication guide: it leaves out other controls and tests.
4. Phases as vertical slices, not modules
I split phases into vertical slices instead of horizontal modules or layers. Each slice delivers end-to-end observable behavior on top of a shared base, has a clear Definition of Done, and can be delegated. I make its dependencies explicit and check that it does not introduce regressions. If a slice goes off track, drift is easier to spot and roll back, though it is not isolated if it shares session state, a database, or contracts with other parts.
In the example, the first slice is “registered user logs in and accesses the service.” Validating the email, sending the code, and verifying it are not slices: they are tasks inside that flow.
5. Visualizing the flow and architecture
A diagram lets me check dependencies and scope faster than prose; I compare it against the acceptance criteria. I use visualization tools, such as my artifacts-manager, to inspect the flow, architecture topology, and components before approving.
In this diagram, the arrows between tasks indicate implementation dependencies: a task builds on what the previous one produced.
flowchart TD
outcome["Outcome: registered user logs in and accesses the service"] --> plan["Plan reviewed by another model"]
plan --> s1["Task 1: validate email"]
s1 -->|depends on| s2["Task 2: send OTP code"]
s2 -->|depends on| s3["Task 3: verify code, create session, and protect the service"]
s1 --> a1(["Agent A"])
s2 --> a2(["Agent B"])
s3 --> a3(["Agent C"])
a1 --> t1["Task 1 tests"]
a2 --> t2["Task 2 tests"]
a3 --> t3["Task 3 tests"]
t1 --> e2e["Integration & E2E"]
t2 --> e2e
t3 --> e2e
e2e --> gate{"Criteria & DoD pass?"}
gate -->|Yes| done["Validated outcome"]
gate -->|No| fix["Fix the failing task"]
fix --> e2e
classDef default fill:#0d152c,stroke:#252f55,stroke-width:1px,color:#f0f4ff;
classDef selected fill:#2e1065,stroke:#c084fc,stroke-width:2px,color:#fff;
class outcome,done selected;
Tradeoffs
None of this is free:
- This workflow also consumes attention when writing criteria, resolving discrepancies, and maintaining checks. Whether it pays off depends on that effort being smaller than the rework it prevents. I do not have data proving that yet, which is why I am sharing it as a practice I am using and evaluating.
- Adversarial review with another model adds token cost and latency. In my case I accept that, because attention costs me more than tokens.
- Short plans mean more cycles. If the outcome is unclear, iterating fast just produces drift faster.
- Test cases only protect what they cover. If the plan, implementation, and tests share the same wrong assumption, they can all agree cleanly and still be wrong.
- Vertical slices cost more upfront when the base of the system does not exist yet.
Next step
If you are spending more time correcting agents today than reviewing results, try this on your next feature:
- Write the outcome in one sentence.
- Translate it into 3 to 5 test cases.
- Ask for the plan and run it past a reviewer using another model.
- Cut it into vertical slices with a DoD.
- Inspect it visually before approving.
The question I still have not settled: how much governance can I move into automated checks before those checks turn into one more thing I have to maintain? Oase classified almost a third of its failed runs (76 out of 241) as deliberate rejections by its own quality gate, and describes that as the gate working as intended. It helps to separate those expected blocks from false positives and from failures in the check itself. I am curious how you are handling that.
