Autonomous Agents Need Boundaries, Not Just Capabilities

The more an AI agent can do, the more it matters what it is allowed to do, and clear limits are what make it safe to let agents act on their own.

6 min readEssay 10 of 13

A company gives a new manager a corporate credit card. The card has a monthly limit. It works for travel and supplies, but not for cash withdrawals. Anything over a certain amount needs a second signature. Nobody takes this as a sign of distrust. The limits are exactly what allow the manager to spend without asking permission every time. Without them, the company would either have to approve every coffee or hope for the best.

AI agents are about to be handed their own credit cards, and many other things besides.

The excitement is about capabilities

Much of the excitement around AI agents focuses on capabilities. Every new release promises that agents can do more. The questions people ask are about what is possible:

Can the agent browse?
Can it write code?
Can it access databases?
Can it execute commands?
Can it deploy applications?

The answer to each of those questions is increasingly yes. But every new capability also creates a new operational risk. Being able to do something is not the same as being allowed to do it, at any time, in any system, at any cost.

Every capability needs a matching boundary

The simplest way to think about this is to pair each capability with the limit it needs:

If an agent can…	…it needs
Execute commands	Boundaries on where and what it may run
Access production systems	Permissions that match its job
Retrieve secrets	Secure credential handling
Spend money on APIs	Budgets
Launch additional agents	Limits on how many and with what access

A few of these need a word of explanation. Production systems are the live systems a business runs on, as opposed to test copies; a mistake there reaches real customers. Secrets are passwords and keys that open other systems. Spending money on APIs means using paid services, including AI models, which charge for each use. And an agent that can start other agents can multiply its own reach, and its costs, very quickly.

An agent that can execute commands needs boundaries. An agent that can access production systems needs permissions. An agent that can retrieve secrets needs secure credential handling. An agent that can spend money on APIs needs budgets. An agent that can launch additional agents needs limits.

The challenge is therefore not simply making agents more autonomous.

It is creating controlled autonomy.

What controlled autonomy looks like

Controlled autonomy means an agent is free to act within a clearly defined space, and the organization always knows where the edges of that space are. A mature agent environment should understand who owns the agent, what it is allowed to access, which tools it can invoke, which actions require approval, what budget it can consume and when its behavior moves outside the expected operating envelope.

An illustration

Imagine a company that uses an agent to reduce its cloud computing bill by shutting down servers nobody is using. The rules below are invented, but show how the pieces fit together.

Owner. The infrastructure team lead, named in the agent’s record.

Allowed to access. Only test and development servers, not the live systems customers use.

Tools. It may pause and restart servers. It may not delete anything.

Needs approval. Any server that has been running for more than 90 days, since it may be more important than it looks.

Budget. Up to 50 dollars a day in AI and service costs.

Operating envelope. Normally it pauses between 5 and 20 servers a night.

One night the agent tries to pause 140 servers. Nothing it is doing is forbidden, and every server is a test server. But the number is far outside its normal pattern, so it stops and asks a person. The person discovers that a new project has put its test servers into the wrong category. Without the boundary, the agent would have shut down a whole team’s work. With it, the problem took ten minutes to fix.

We have solved this before, for people

This mirrors the evolution of human access management, the way organizations decide which employees can use which systems.

Employees are productive because they have tools and permissions. A new accountant who could not open the accounting system would be of little use. But those permissions are structured around roles, policies and accountability. The accountant can see the ledgers but not the payroll of the executive team. Large payments need a second approval. Every change is recorded with a name next to it.

AI agents will require the same concept. Each agent should have a role, and its access should follow that role. Its actions should be recorded, and there should always be a person who is accountable for it. The difference is speed and scale. An agent can take thousands of actions in the time a person takes one, so boundaries need to be enforced automatically rather than checked afterwards.

What this means for your organization

Write down what each agent may do. For every agent that can take real actions, list what it can access, which tools it can use and what it must never do.

Name an owner. Every agent needs a person who is responsible for its behavior and can switch it off.

Set a budget. Give each agent a spending limit for AI and other paid services, with an alert when it approaches the limit.

Choose the approval points. Decide which actions are important enough to need a human yes, such as deleting data, making payments or changing live systems.

Watch for unusual behavior. Record what normal looks like for each agent and flag anything far outside it, even if it is technically allowed.

Autonomy without boundaries creates risk. Boundaries without autonomy eliminate most of the value.

The architecture must support both.

Tracston works on these questions in Otopia.