5 October 2026 · 2 min read
What agents change
A model that answers is a tool. A model that acts is a colleague you have to supervise. The difference is one loop, and it changes the arithmetic of reliability.
A chat model takes a question and returns an answer. An agent takes a goal and works toward it: it plans a step, acts, looks at the result, and plans the next step. The loop is simple to describe. Its consequences are large, because a system that acts can finish work that nobody has time to supervise line by line.
The parts of an agent
- A model that decides what to do next.
- Tools it can call: a search engine, a code interpreter, a browser, a company's internal systems.
- Memory of what it has already tried, so step forty knows about step three.
- A stopping rule: done, stuck, or out of budget.
Why long tasks are hard
Errors compound. Suppose an agent gets each step right 95 times in 100. Across a task of twenty steps, the chance that every step is right is 0.95 multiplied by itself twenty times, which is about 36 percent. A system that looks excellent on single questions fails most long tasks.
That arithmetic explains where the effort goes. Raising per-step reliability from 95 to 99 percent lifts the twenty-step success rate to about 82 percent. Small gains per step become large gains per task. It also explains why the length of task a system can finish alone is one of the better measures of progress, as argued in How do you measure a mind?
What supervision looks like
An agent with tools can change things in the world, so the question of oversight stops being abstract. Good practice is borrowed from how organisations handle new employees and powerful software.
- Give the least access that lets the job get done.
- Keep a log of every action that a person can read afterwards.
- Require human approval before anything that cannot be undone, such as sending money or deleting data.
- Make it easy to stop a run halfway.
Why this matters for superintelligence
Research is a long task. An agent that can plan experiments, write the code, read the results and decide what to try next is doing the work described in the intelligence explosion argument. Agents are the practical route by which AI systems start to contribute to AI research. Watching how long and how unsupervised their work becomes is watching the loop begin to close.