Dear Reader,
For three years, most enterprise AI has advised. It drafted the email, scored the application, summarised the contract, and a person decided what to do next. The output waited until someone used it. Agentic AI changes the job. The same model that used to draft a rejection letter now rejects the claim itself; the one that flagged a payment for review now releases it. Instead of producing something for a person to act on, the software takes the action.
This is the line that matters in enterprise AI, more than which model you run or how it is built. When the software advises, a human is always in the loop, because advice does nothing until a person picks it up and acts. When the software acts, that human is gone unless you put them back. This is the first of several issues on what changes once agents act, and the rest of the series rests on this one distinction.
Why acting is a different kind of risk
An AI that advises and gets it wrong costs you the time to notice and discard the answer. The person reading it is the filter, and a wrong recommendation travels no further than their judgement. An AI that acts and gets it wrong has already done the thing. The money has moved, the claim is closed, the customer has been told. There is no draft to fix. The action has already happened.
Three things change the moment the action is real.
The first is speed. The human used to be a safeguard simply by being slow: before a decision was carried out, there was always time to look at it. An agent works at the speed of software, so the action is done before anyone has looked at it.
The second is reversibility. Advice can always be undone, because you can ignore it. An action often cannot. A transaction held for review can be released later; a sum that has left the account has to be clawed back, if it can be at all.
The third is accountability. When a person acted on AI advice, that person answered for the decision. When the agent acts, who answers for it is a question most organisations have not settled.
The review step that was never there
In Issue 33 I wrote about making human oversight meaningful: how a review step decays into a rubber stamp, and how to design it so the person actually engages rather than clicks approve. That issue assumed the review step existed. An agent that acts on its own raises the prior problem. There may be no review step at all.
The agent moves from one action to the next without stopping. Unless someone built a pause into the process, there is nothing for a human to be meaningful at. In Issue 33 the problem was a weak review. Here it sits one step earlier: by default, there is no human in the process at all.
Two kinds of error that appear once AI acts on its own
The first is a wrong decision made with full conviction. We already knew models can be wrong while sounding completely sure of themselves. While the model only advised, a person usually caught it. Now the model carries out that wrong decision itself — and there is no one left to catch it.
The second is rarer and worse: the half-finished action. An agent runs a sequence of steps, and something stops it partway, because a system it depends on goes down or a limit is reached. A person interrupted mid-task leaves a note and comes back to it. An agent can leave a process in a state no one designed: the payment sent but not recorded, the policy cancelled but the customer never told. The work is neither done nor undone, and often no one knows it is sitting there.
This is an old problem in software engineering. Databases solve it with atomic transactions: either the whole operation completes or none of it does. It is harder with an agent, because it works across several systems that share no single transaction, and it improvises the order of the steps as it goes — so it does not get that guarantee, and you cannot simply bolt it on afterwards. Better-built systems at least narrow the risk: they reserve in advance the resources a task will need, so it does not die halfway.
The rule is reversibility
Whether you can undo what an agent has done, without loss, is the test that should decide how much it is allowed to do on its own. That matters more than how capable the underlying model is.
It shows up most clearly at the extremes. Even an agent that is wrong once in a thousand times should not, on its own, take an irreversible step with significant consequences — while a mediocre one can run free wherever a mistake is cheap to undo.
In practice you sort an agent’s actions by that one criterion. The reversible ones, or the cheaply correctable, it does itself, with a person watching only the broad patterns. The irreversible ones, or the hard-to-reverse, wait for that person’s approval before they happen. It is the human-in-the-loop and human-on-the-loop distinction from Issue 33, except the line now runs somewhere you can point to rather than only feel.
Two agentic systems I assessed this quarter, both in regulated industries, are built around exactly this — in two very different domains. In one, the agent works in claims handling, say: the human checkpoint sits where money irreversibly leaves the firm, not in a dashboard someone scans the next morning. In the other, agents write and ship software: the irreversible step is what goes to production, so a person decides what gets deployed and the agent handles the rest. Same principle, two different points of no return. In both, a record of what the agent did is written as it works, because when something goes wrong the first question is “what happened?”, and the answer has to already exist. And the model is given as little to decide as possible: the routine, rule-bound steps run as ordinary code, and the model is brought in only where the work genuinely needs judgement, so the agent behaves predictably. None of this shows in a demo. It is the difference between an agent that demos well and one you can let act on a real customer’s money.
Read in isolation, the checkpoint looks like a brake on something meant to be fast. In fact it is what makes the speed safe: the pause in front of the irreversible action is what lets you give the agent a free hand everywhere else. Take it out and you are not faster, only more exposed on the day the agent is confidently, expensively wrong.
The regulation that matters here is about what the system does to people, not about how it is built. GDPR gives a person the right not to be subject to a decision made solely by automated means when it significantly affects them. The EU AI Act requires, for high-risk systems, that a human can oversee the system and step into what it does. Neither asks where your code runs. Both ask whether a person can stand between the machine and a consequential action. In Poland that is UODO’s territory on the data-protection side, and KNF’s once the agent acts inside a banking or insurance workflow. An agent that takes decisions affecting people with no one able to step in is a compliance problem before it is anything else.
Where autonomy ends is the decision a leader has to own. It is the one that decides what happens on the day the agent gets it wrong, so make it deliberately, rather than letting the system’s defaults settle it for you.
Briefing
OpenAI released its new flagship model family, GPT-5.6, but only as a limited preview to around twenty government-approved partners, after the White House asked the company to restrict distribution over the model’s capabilities, the same treatment recently given to Anthropic’s “Mythos” (CNN). Access to the frontier is starting to be gated in Washington, which is a new variable for any continuity or model-sourcing plan that assumes you can simply buy the best model available.
Qualcomm agreed to buy Modular, the company behind the Mojo language and the MAX inference stack, for nearly $4 billion in stock, aiming squarely at the software moat that keeps Nvidia’s chips hard to leave (WIRED). A credible second source of inference hardware would, over time, loosen a dependency most enterprises do not realise they have.
Large enterprises are starting to rein in AI spend and shift toward cheaper and open-source options, putting pressure on OpenAI’s and Anthropic’s growth as both head toward public markets (CNBC). The cost-per-outcome conversation has arrived, and CIOs now have analyst cover to renegotiate model contracts rather than default to the frontier labs.
Questions for your leadership team
- For each AI system we run, do we know which actions it can take on its own without a person, and did someone decide that deliberately, or did it simply accumulate?
- Which of our agents’ actions cannot be undone — money moved, a contract issued, a customer notified — and is a human in front of each of those before it happens?
- If an agent failed halfway through a task tonight, would we know it had left something half-done, and who would put it right?
- When an agent acts wrongly, who here answers for it? Have we written that down, or will we work out the answer during the incident?
Summary
Enterprise AI is crossing the line from advising to acting. While it advised, a human was always there to catch a wrong answer. Once it acts, the error is the outcome, and the human is gone unless you put them back. The rule for where to put them is reversibility: let an agent act on its own where the action can be undone, and require a person’s approval for the actions that cannot be undone cheaply. The systems already doing this in regulated work place the human checkpoint at the step that carries the consequence, keep a record as the agent works, and let the model decide only what genuinely needs judgement. Where that autonomy ends is the line a leader has to draw.
Stay balanced, Krzysztof Goworek
Krzysztof Goworek is founder of Quintant — AI advisory that gets enterprises from experiment to production value.