What building two agent studios taught me about trusting machine work
Getting a machine to produce something is no longer the hard part. The hard part is deciding when to stop steering and leave it alone, and knowing whether the thing it just handed you is real or just well presented.
I came to AI agents from specialist retail and digital marketing, not from software engineering. I have spent years in the world of selling things, getting them found, and understanding what a customer actually wants. Over the last year I have also built and now run two multi-agent systems on real businesses: the Foundry that runs the content and digital work for ebikeist, a premium e-bike retailer, and Hyde Studio, the system I wrote up in the first of these posts. Coming at it from that side, the thing I keep bumping into is not the question most people expect.
Trusting machine work is the skill of knowing when an AI agent's output is good enough to act on, and when it only looks finished. Getting a machine to produce something is no longer the hard part. It will always produce something, confidently and quickly. The hard part is deciding when to stop steering and leave it alone, and knowing whether the thing it just handed you is real or just well presented. That judgement takes practice and experience.
The hard part moved
The clearest writing I have found on this is by Nate B Jones, who frames the current tools as two schools for managing machine work. One trains you to steer: stay close while the problem is still forming, shape it as it goes. The other trains you to dispatch: send out a bounded job and demand receipts when it comes back. I work mostly in the first mode, close and conversational, and that is where I have learned the most about what goes wrong.
What goes wrong is that the machine gets good at the appearance of the thing before it gets good at the thing. Nate has two names for this that I wish I had had six to twelve months ago. Understanding theatre is when a good conversation convinces you the work was understood, because the agent said all the right words back to you. Completion theatre is when a finished run feels more done than it is, because there is output sitting there and output reads as progress.
How do you know when to trust what the machine hands back?
You assume it is performing until you spot otherwise. In practice that means building systems that make the machine show its working, not just its conclusions. Here is what that looks like in mine, concretely.
My studio opens every session by running a script that outranks my own memory. I do not get to claim what day it is, or what state things are in. The script walks the project, checks the clock, and tells me, and if my memory and the script disagree, the script wins. It sounds small. It is the difference between a system built on what I think happened and one built on what did. And it stops my mind secretly dodging stuff that it doesn't want to do!
Every piece of work becomes a numbered entry in a findings tracker, with one rule that does the important job: delivered is not done. A row stays open until the action is actually finished, not when an artefact exists. An agent producing a document does not close the job. That single rule is the whole defence against completion theatre, written down so I cannot talk myself out of it on a busy day.
Anything a report-maker produces has to pass a completeness gate. If a required part is missing, it refuses to declare the thing ready, names the gap, and stops, rather than papering over the hole with confident prose. A write-up that leaves out the bit that did not work is not a write-up, it is an advert. A gate is only real once I've watched it fire.
And anything that acts in the world, sends a message, spends money, changes a setting, waits for my explicit go. No agent does those on its own. The machine drafts; I decide.
What is next
The next pass is to turn that same "is it real" lens on my own studio: which of my agents hand back inspectable proof, a file path, a diff, a list of sources, and which hand back a confident paragraph I have been taking on trust. I will write that up too, including wherever it turns out I have been fooling myself.
Alongside it, the products. The current one is a client app for a personal trainer.
If any of this is useful, or you are building something where understanding a business, marketing it, and actually shipping the software all have to meet in one place, I would love to hear from you.