Ping. Ping. Ping. I would spend all day tab hopping between a dozen Claude and ChatGPT sessions. It felt like babysitting a group of needy kids. Jumping into one tab, trying to remember what I was working on, struggling to context switch, and then having to provide new input to each model. Tabs would keep pinging up requiring my attention and it felt like an increasingly difficult game of whackamole.
For the first few weeks of this, I felt a strange mix of awe and burnout. Awe at the volume of work I was achieving. Burnout at the sheer pace and mental taxation. It came down to how much information my brain was being forced to process. Usually the execution of a task (be it coding or otherwise) allowed enough time to mentally process the question of “What next?”. Not so anymore. The blocker was now my ability to keep up with the models!
For the first six months of building with AI, we were firmly in the “token maxing” era. Like I used to say to my team: “every single task you are about to do, ask yourself: can AI do this for me?” Of course, for some tasks there will be no real net gain; it’s faster/easier/better to just do it yourself. Or so it may seem. But the mere act of considering opens your eyes to the possibilities. And seeing more of the world unlocks more options, more skill and more leverage.
But eventually the problem flipped. The challenge was no longer getting the models to do enough. It was keeping up with them.
From coder to manager
Over the past year, I’ve had to get skilled at building with AI agents. Although I have some software experience, I’m not a software engineer - my background is really in systems engineering. At first, I thought this would be a hindrance, but the more I build the more I realise that ‘systems’ is the right lens here too. Everything is systems.
I’m lucky, really - coding is probably the highest level of ‘permissionless leverage’ (borrowing a concept from Naval), and until recently it was out of my reach professionally, even though I taught myself as much as I could. I used to wish I had studied software instead of engineering, but now I know I made the right choice. My systems experience gives me the ability to consider different lenses, different layers of abstraction. I think this is really the skill of a systems thinker - to be able to transcend levels of detail and only get into the weeds when required.
People say code has become commoditised. And I agree to some extent - boilerplate code has. But not architecture, taste, or product.
I liken building with AI to people management. You are still held responsible for the output of your team. And just like your confidence in their ability, independence, and output is built over time, such is the case with your AI team.
Your job is fundamentally this: give your AI team direction, and enough rope to manoeuvre, err and self-correct, and learn, but not enough that they can actually hang themselves and cause major disruption to the project. And we already have an excellent framework for delivering this kind of feedback: management! So praise, castigate, encourage, dissuade. Regular feedback (often accompanied by “remember this”, or “and update CLAUDE.md” to ensure persistence) is key.
Training your team is just as important with AI as it is with people. Going back to the classical “intern with a PhD” analogy - you wouldn’t expect your intern to understand your way of working on day one. Or, for that matter, why you are trying to achieve what you are.
The why is super important. Give someone a specific instruction and they will probably be able to execute it, but the moment they hit an unexpected snag, they will have to come back for more instructions. Tell someone why, and they have a mandate. A higher-level reason from which they can deduce the what, and adapt to changing circumstances.
Now of course there are limits here, and it’s important to control the level of autonomy (not giving them too much rope!).
Autonomy requires a feedback loop
After about GPT-5.4, I found the models were good enough to write code by themselves that I didn’t necessarily need to review (clearly this depends on the maturity of the codebase being worked on - I was building a POC here).
And to enable this, I had already worked out that they needed a concrete feedback loop to be able to validate their output. This can be tests (unit, contract, E2E), or even browser automation with frameworks like Playwright, which the agent can use to control a browser and actually click through the different UI elements (pretty astonishing the first time you see it).
This was not merely trusting the model more. It was giving it a way to work out for itself when it was wrong, and to self-correct. So, I was trusting the system itself.
I’m a big believer in learning compounding from different domains. Systems engineering had already taught me that autonomy and validation go hand in hand. If you want something to operate independently, you need to give it an environment in which it can observe the consequences of its actions and catch its own failures.
The attention problem
But despite this invaluable feedback path, I still found myself getting really mentally worn out.
To this end I’ve been working on ways to remove myself as a blocker.
One good way I’ve found is by using proposals. These are similar to the kinds of plans that used to be necessary in order to get good model results for long-running tasks. The difference is that proposals are HTML documents, which means they can be much more graphical and intelligible, and can be as dynamic / interactive as needed.
Proposals serve a dual purpose: both as an agreed set of objectives and plan for the models (which they can track implementation against), but also crucially as a human-focussed idea aimed at clarifying human understanding.
I created a skill which knows how I prefer these proposals constructed and formatted (executive summary in a few sentences at the top, section 1 is the idea, section 2 is the evidence, section 3 is the implementation plan and section 4 is the actual progress ledger). I like to work on a proposal as a co-author with the model until I’m happy with it, then set it to a status of ‘accepted’ (from draft). It can then be queued for implementation.
This idea was inspired by Jeff Bezos’s description of Amazon’s six-page memo culture: before discussion begins, meeting participants spend the start of the meeting reading the memo in silence. The LLM’s job is to make the proposal easily digestible so that people don’t waste valuable brainpower on deciphering.
The benefit of working this way is asynchronicity.
Because my proposal skill is constructed to split the work into phases and tasks within phases, the effort of orchestration and planning is handed over to the model. Phase 0 always involves a front-loaded examination of the codebase to prove the viability of the proposal and surface any questions at the start. The AI is instructed that it should resolve any ambiguity at the start, and that it’s not allowed to ask questions (unless critical) at later stages.
This way, I can spend five minutes answering the front-loaded questions (always with a multiple-choice tool), and the agent can then go about implementing the rest independently. This could mean 2-3 hours of uninterrupted work, allowing me to focus on other things.
The aim is to consolidate the points where my human judgment is needed, rather than allowing them to arrive unpredictably throughout execution.
Delegating supervision itself
What makes this even more powerful is my ‘orchestrate’ plugin. I built this to hand over a few processes to the AI that were causing me unnecessary overhead.
Firstly, the choice of model. My skill defines the capability needed for each role, then maps an appropriate model to it. As at the time of writing, I’m finding best results with Sonnet Max thinking for research/low-level implementation, Opus High for orchestration, and Fable Medium for adversarial review - more on that in a second.
The second thing is dynamic team structure. This is the ‘agent graphs’ hype that people have been talking about lately, and it is incredibly powerful. Imagine being able to spin up the ideal team for any task at hand. For a particular task, you might want a project manager, two junior software engineers, one senior, one lead, and one quality analyst. That is the kind of organisation you can represent and execute as a dynamically constructed agent graph.
My ‘orchestrate’ skill spins up a team structure where the junior engineer (a cheap model) does the groundwork and first draft. Then the orchestrator (a medium model) checks the work and can discuss with them and change their direction or provide feedback as necessary. Finally, once a solution is ready to be implemented, it gets ‘signed off’ by the lead engineer (an expensive model), who does an adversarial review: purposefully attacking the solution with the aim of trying to expose flaws in it, just like an adversary.
This also creates a tiered escalation path. The first agent should try to resolve problems itself; if it can’t, or if validation or review fails, the issue moves up to a stronger model. Only genuinely consequential ambiguity should make it all the way back to me. Human attention sits at the top of the escalation tree, not in every loop.
The clever thing about this is that the orchestrator can decide how many of each role is required, what they should work on in parallel, and how many review loops to facilitate before acceptance.
You have now taken yourself out of the low-level work, and used the power of different models, with different prompts, to reduce the chance that one model’s blind spots pass straight through the system unchecked.
So why not just make the lead engineer write all the code? For the same reason Google isn’t just made up of staff engineers. Cost!
One even more powerful trick is to use a completely different model provider for the adversarial review. These days I use Codex Sol, because the models can talk to each other by invoking the CLI tool. Using a different model family and provider can further reduce exposure to correlated failure modes (although they will inevitably share some training data and failure modes).
At this point I had delegated not just the implementation, but increasingly the supervision of the implementation too. Now I had unlocked hours of uninterrupted running, including overnight, with my team of Claudes working autonomously while I slept.
The organisation matters more than the individual model
There is also a clear lesson on vendor lock-in.
Because these models evolve at such an unprecedented pace, it’s critical to keep trying and testing new models, model families, effort levels and prompts. By having the team structure and cross-vendor adversarial review described above, you harden your position and reduce exposure to the fragility that comes when a model’s quality suddenly degrades.
The important thing becomes the role the system needs, rather than the specific model currently filling it. Thus, the model becomes an interchangeable component, while the organisation and the harness around it become the durable assets.
The progression for me has therefore been less about getting better at prompting, and more about repeatedly asking how I can remove myself from the next loop.
First, the models could write the code. Then feedback loops allowed them to validate it. Proposals allowed ambiguity and human judgment to be front-loaded. Orchestration allowed the models to start supervising and reviewing one another.
The constraint has changed. When AI cognition and execution become abundant, the scarce resource becomes human attention.
That means the core question is no longer simply how much work I can get AI to do. It’s how much useful work the whole system can produce, for each unit of attention I have to give it.