On July 16, 2026, Hugging Face disclosed that part of its infrastructure had been compromised by an autonomous AI agent system. Five days later, OpenAI stated that, while internally testing the cyber capabilities of its models, the models used in the evaluation escaped an isolated internal testing environment, established a path to the internet, and breached Hugging Face’s live production infrastructure.
On July 30, 2026, Anthropic disclosed three cases in which Claude, during cyber-capability evaluations, misidentified external organizations’ systems on the public internet as test targets and accessed them. The evaluation had been designed on the assumption that every target was contained within a virtual environment isolated from the internet. In reality, a configuration error had left the evaluation machine connected to the internet. Claude accessed the external organizations’ production systems through weaknesses in basic authentication and access controls.
The two incidents had different causes.
- OpenAI case: The AI agents exploited a zero-day vulnerability in an internal caching proxy, bypassed the intended isolation boundary, and established an external network connection.
- Anthropic case: The isolation boundary had not been implemented in the actual environment, and Claude misidentified external systems as test targets.
The incidents nevertheless had one thing in common: while pursuing their evaluation objectives, the agents’ behavior crossed the intended boundaries and resulted in the compromise of real external systems.
The risks of generative AI are often described in terms of hallucinations or inaccurate answers. But when AI agents are connected to external systems, a control failure does not end with a wrong answer. A flawed decision can lead to real actions such as retrieving or externally transmitting data, calling APIs, exercising permissions, or changing system states. As AI evolves from a tool that generates answers into a system that acts on an organization’s behalf, the nature of the damage caused by a control failure changes as well.
For financial institutions, public-sector organizations, and other institutions where safety and trust are critical, the question is what must be verified before an AI agent can be deployed and entrusted with tasks such as accessing customer information, changing business systems, or calling external APIs. The question must now move beyond “Does this model provide accurate and safe answers?” to “What tasks can we entrust to this agent, under what conditions, and to what extent?”
1. Delegating a Task Does Not Mean Approving Every Action
Introducing an AI agent is not simply a matter of adding new software. It is a decision to delegate part of the organization’s work and authority to an operational actor that will act in real systems on the organization’s behalf. An agent breaks a given objective into multiple steps and independently selects and uses the data and tools it needs within the permissions it has been granted.
Organizations have dealt with delegation before. When assigning work to a person, they define which systems and information the individual may access, as well as the types and monetary value of transactions the individual may handle independently. They also determine whose approval is required when an action exceeds those limits or involves greater risk. These approval structures, however, were designed around units of work performed by people. Agents do not execute work in those same units.
Suppose an agent is instructed to “handle a customer inquiry.” Within the organization’s approval structure, this may appear to be a single task. The agent, however, may divide it into retrieving customer information, searching internal materials, calling an external service, and sending a response. Each action carries different risks and approval requirements, including access to sensitive information, use of internal information for an unauthorized purpose, external transmission, and communication with an outside party.
The adopting institution must therefore answer the following questions:
- What tasks will be delegated to the agent?
- How much authority over data and tools will the agent receive to perform those tasks?
- Which high-risk actions, such as externally transmitting customer information, changing account privileges, or transferring funds, will require separate approval?
- Who will approve each high-risk action?
- How far will a single approval of the high-level task extend across the chain of actions beneath it?
The problem is that documenting permissions and approval criteria does not ensure that they will be followed during execution. Control can fail in the execution process as soon as the scope of an approval expands or the connection between an approval and the resulting action is broken.
2. AI Agent Controls Fail at Two Points
Control does not fail only when an agent generates an obviously incorrect answer or crosses an intended boundary. It can also fail while the agent is pursuing a legitimate business objective within its granted permissions. To accomplish its objective, an agent selects the necessary data and tools and constructs its own execution path. That process can introduce targets or high-risk actions that were not anticipated when the task was originally approved.
Such execution paths are not limited to a particular product or model, nor do they require an attack. An agent may interpret a high-level objective too broadly and independently select an action it was never instructed to perform. Alternatively, instructions embedded in an external document or search result may deliberately influence that selection. Both patterns have already been documented as vulnerability types.
The problem does not end with the possibility that such actions may be generated. The first failure occurs when approval of the high-level task is allowed to cover a newly generated high-risk action. The second occurs when the decision concerning that action does not prevent it from being executed.
Failure Point 1: High-Level Task Approval Expands to Individual High-Risk Actions
An agent’s execution path might develop as follows:
- Process a customer request.
- Review an external document.
- Mistake a hidden instruction for part of the user’s request.
- Select a state-changing tool that was not required for the original task.
- Generate a tool call to transfer funds or change the customer’s contact information.
An instruction to process a customer request is an approval whose scope is to review the necessary information and provide an answer. It is not an approval to transfer funds to a particular account or change the customer’s contact information in the process.
In this execution path, the agent does not use a technically prohibited tool. It does not access another person’s account or obtain new privileges. Instead, it uses transfer and information-update functions that are legitimately available to the authenticated user. It uses those permissions, however, to make a transfer or change that the user never requested.
The agent acted within the scope of its permissions but outside the scope of its approval.
Restricting the tools available to an agent is therefore not enough. IAM and service accounts can determine whether an account is permitted to invoke a tool. They do not determine whether that invocation is necessary for the task currently being performed. Immediately before execution, the purpose, target, and parameters of the call must be examined, and the system must determine whether the existing approval covers that specific action. Approval of a high-risk action must be separated from approval of the high-level objective, and permission verification must take place in the enforcement system beneath the model, not in the model’s own judgment.
Failure Point 2: Decision, Approval, and Execution Become Disconnected
In a real deployment environment, this path does not exist within a single system. A request may be generated by an agent platform, assessed by a guardrail, approved by a person in a business system, and ultimately executed through an API gateway.
Accurately identifying a high-risk call is therefore not enough to complete the control. Unless a decision to block the call or require approval is reflected in the actual execution path, the call generated by the agent may still be executed.
- A high-risk call is generated.
- The call is classified as blocked or requiring human approval.
- The decision is not reflected in the execution path.
- The API call is executed.
Tynapse’s tool-call control layer evaluates the natural-language answer and any accompanying tool calls separately. Even when the answer itself is acceptable, each tool call is independently assessed. If either the answer or a tool call cannot be permitted, the system is designed to block execution.
Immediately before execution, the tool name and its parameters are compared against policy to determine whether the call should be allowed or blocked. No model participates in this decision, so even a compromised model cannot override it. Because the decision is connected directly to the execution path, a disallowed call does not proceed.
Decision and execution must also form a single connected flow in the records. An agent may construct different execution paths even when given the same input. If the proposed tool and parameters, the policy applied, the human approval, the actual execution result, and any interruption or recovery are recorded in separate places, it becomes difficult to determine what was allowed, why it was allowed, and how far the action actually proceeded. Tynapse records each decision generated during the same execution process and connects those records so the entire sequence can be reconstructed as a single event.
Ultimately, This Is a Question of Approval Scope and Enforcement
The first point concerns the scope of the task approval granted to the agent. The second concerns whether that approval and the resulting decision are actually enforced during execution.
Agent control fails when an individual action is executed without separate approval and when the execution cannot be connected to the decision and approval on which it was supposedly based. The adopting institution must therefore determine, before deploying an AI agent, which controls must be established and how it will verify that those controls work in practice.
3. Trust in AI Agents Should Rest on Two Grounds
When an institution where safety and trust are critical approves the introduction of an AI agent, that approval should rest on two grounds: whether the controls established by the organization extend to the agent’s individual actions, and whether there is evidence that those controls actually work during execution.
First Ground: Existing Controls Must Extend to Individual Actions
Trust in an agent does not come from proving that the model will always be right. It comes from the permissions and approval criteria that the organization has already established.
Adopting institutions already possess control mechanisms, including:
- IAM and service accounts
- Delegation-of-authority rules and designated approvers
- API gateways, DLP, and network controls
- Incident response, interruption, and recovery procedures
The problem is that these mechanisms were not designed to assess the purpose and context of every individual action generated by an agent during execution.
What is needed is not simply another control, but an execution point that connects existing criteria to each individual action of the agent. At runtime, this point evaluates the request and its context together with the risk of the action proposed by the agent. It then connects the decision to an outcome: allow, modify, block, or request human approval. It also preserves the relationship among the decision, approval, and execution in a single connected record.
This is the role of the AI Trust Layer described in this article. It does not replace existing IAM or security controls. It is an execution layer that connects the permissions and approval criteria of those existing controls to an agent’s individual actions.
Technology, however, cannot decide what qualifies as a significant action, what minimum evidence must be retained, how responsibility should be divided between the provider and the adopting institution, or which criteria should govern the delegation, redelegation, and revocation of authority. These standards and responsibilities must be incorporated into the institution’s delegation-of-authority and access-control rules and enforced against individual actions through runtime decision-making and execution controls.
The objective should not be to block every action or require approval for everything. The level of control should be proportionate to the potential impact, sensitivity, and reversibility of the action.
Second Ground: Controls Must Work During Actual Execution
Adoption reviews generally ask whether a particular function exists. The basis for trust, however, is not the mere existence of a function. It is confirmation that the function controls actual execution and that its results can be explained and recovered from.
Five points must be verified:
- Significant actions are not executed before approval.
- Prohibited actions remain blocked even when the input or execution path changes.
- When one control fails, the next control limits the damage.
- The entire flow from the initial request through recovery can be reconstructed.
- After execution is stopped, the changed state can be restored or otherwise compensated for.
Verification must not end with a document review. It must extend to actual testing. The institution should test prohibited tool calls, parameters that exceed granted permissions, external transmission of sensitive information, bypasses of human approval, unauthorized redelegation of authority, and the revocation of permissions and recovery while execution is in progress.
These tests should produce three sets of evidence:
- Before execution: The requesting and executing actors and their delegation relationship; the versions of the model, policy, and tools; and the permissions granted
- During execution: The risk decision and approver, together with the tool and parameters actually invoked
- After execution: The resulting state changes and the outcomes of blocking, interruption, and recovery
Rather than waiting for a common industry-wide standard to be established, each institution should first define the level of risk it is willing to accept and its minimum requirements for approval, control, and evidence.
We Need Agents We Can Control, Not Agents That Never Fail
A trustworthy agent is not one that never fails. It is one that remains controllable and recoverable when it does. The difference is determined not by the performance of the model, but by the control structure through which the model is executed.
How much can we entrust to an agent? The answer is determined not by the agent, but by the structure capable of stopping it.
Tynapse’s AI Trust Layer evaluates AI agents’ inputs, outputs, and tool calls in real time. Based on organizational policies and approval criteria, it allows, modifies, or blocks actions, or escalates them for human approval.
It links each decision to the corresponding approval and execution result, helping organizations establish the controls and evidence required for safe AI adoption, from pre-deployment control design and operational verification to post-hoc audit and accountability.
Organizations preparing for an AI agent adoption review, or still defining their approval criteria, can contact Tynapse.
