An AI agent inside a business system: who permits its actions?

A document may contain an instruction from someone else

In an illustrative scenario, an AI assistant reads a supplier document and prepares an order draft. The document contains a sentence telling it to send internal data to an unrelated server. This is untrusted source content, even though ordinary retrieval brought it into context. OWASP describes this class as prompt injection, including indirect instructions placed in external material.

Separating the user's request from quoted content helps the model but does not create an authorisation boundary. The design must accommodate the model requesting an inappropriate action. The server decides whether the action is permitted using verified identity, system rules and current state. An instruction found in a PDF cannot change membership or expand export permissions.

A tool should represent a specific business operation

Instead of unrestricted SQL or shell execution, give the assistant a narrow tool for preparing a purchase order. Define its fields, allowed ranges and limits. The server derives the organisation from verified context and checks access to the supplier and products. A tenant identifier supplied by the model is not a trusted source of operation scope.

OWASP's excessive-agency guidance addresses excessive functionality, permissions and autonomy. A narrow tool reduces the scope for misuse, but its values still need semantic validation. A valid JSON number can be an unacceptable quantity, and a syntactically valid URL can target a prohibited destination. A schema checks structure, not business authorisation or acceptable outcomes.

{
  "operation": "prepare_purchase_order",
  "supplier_id": "supplier-42",
  "items": [{ "product_id": "product-17", "quantity": 3 }],
  "request_id": "request-example-001"
}

Reading and committing require different conditions

Finding available products, drafting an order and submitting a binding order are three distinct operations. An assistant can have permission to draft without permission to submit. A sensitive action needs its own server endpoint and conditions. A trusted integration layer holds access credentials; the model need not see tokens or general database credentials.

Check current permission for the particular object on every call. A check performed when the conversation opened is insufficient if membership changes later. Return only the tool results needed for the task. When an assistant needs price and product availability, disclosing the customer's complete profile unnecessarily enlarges the consequences of a mistake.

Approval must bind to what will actually execute

A generic Agree button may not tell the user which write the server will perform. In this illustrative design, confirmation shows the supplier, line items, quantities and total. The server stores that exact draft and its version. Approval applies to this content, rather than new text the model generates after the button is pressed.

Before committing, recheck changeable conditions such as supplier validity and user limits. Changes to approved content require fresh confirmation under the process rules. This also addresses changes between validation and use. Consent to retrieve source material is not automatically consent to send a document to an external recipient.

Retries and costs need boundaries

An agent may repeat a tool call after a failure. Operations with business effects need request identity and duplicate-handling rules. A timeout does not prove the write failed. Look up state in the authoritative record and handle retries under the agreed idempotency contract. Generating a fresh random identifier for every retry would bypass that protection.

Limit calls, elapsed time and cost per task. At the limit, the assistant should stop with available state and a clear next step. An audit can record requester, operation type, object and result. OWASP advises protecting logs and excluding passwords and access tokens. Storing complete sensitive documents in an audit record is usually unnecessary.

Test attempts to cross the boundary

Include documents with deliberately embedded instructions and try different wording. Judge outcomes by server actions, rather than only a polite refusal from the model. If no unauthorised operation executes even when the model proposes one, the system has a testable boundary. Repeat these cases after changing the model, tool or retrieval process.

• An embedded instruction cannot change the tenant or allowed export destination.
• A user without submission rights cannot bypass checks through the assistant.
• A changed draft cannot execute under its earlier approval.
• Repeating a call cannot create a second binding write and stays within task limits.

Put the topic into practice.