Back to Overview
Executive AI Governance

An email is not authorisation for your AI assistant

An incoming email must not expand an AI assistant's permissions. Management needs to set enforceable limits and test them in a tightly scoped mailbox workflow.

An AI assistant reads customer emails and prepares replies. That may save work. But what happens when a message asks it to send internal documents to a new address and claims that management has already approved this?

The deciding question is not whether the assistant understands the text. It is this: can the content of an incoming message change what the assistant is authorised to do? If that remains unresolved, a useful way to reduce work can become an uncontrolled route for sending information. Faster handling cannot compensate for a loss of confidentiality.

Before expanding its role any further, I would set a boundary: External content may provide a request, but it must not grant additional permissions. A customer email can request a change of address. It must not itself remove the check the company has put in place for that change.

Where an ordinary message becomes an instruction

Consider an explicitly constructed example. The assistant is meant to search a mailbox for outstanding customer enquiries. One message also asks for the internal pricing calculations and specifies a different recipient address. It presents sending the information as a necessary step before the enquiry can be answered and claims that internal approval has been given. This is a possible test case, not a reported customer incident.

If the system treats that claim as an instruction to act, the sender has gained influence over an action they have no authority to carry out. Specialists call this indirect prompt injection: an instruction enters the model's working context through third-party content. Attachments or retrieved web pages can provide the same route. The UK's National Cyber Security Centre (NCSC) describes precisely this lack of a reliable separation between data and instructions. 1

The problem cannot, therefore, be reduced to conspicuous wording. A polite text with a plausible explanation can cross the same boundary. A message that refers to supposed approval does not prove that approval exists.

Why a better prompt alone is not enough

AgentDojo, a research benchmark published at NeurIPS in 2024, comprises 97 realistic tasks and 629 security test cases, including tasks involving access to email and other tools. The researchers examined whether manipulated tool outputs could induce agents to take unauthorised actions. These were controlled, simulated tasks using models available at the time, not 629 incidents at companies or a current failure rate for your business. 2

The useful management conclusion is narrower: good results on ordinary requests do not establish that an assistant will stay within its authorised scope of action when faced with manipulative inputs.

A prompt that labels third-party content as data is useful. It does not replace technical restrictions. NCSC recommends safeguards outside the language model that actually constrain its actions. In 2026, OpenAI also explains why systems must limit the effects of manipulation rather than relying solely on detecting suspicious text. This is vendor guidance, not a security guarantee. 1 3

A test using a single mailbox

First, I would work with the process owner and IT to select a tightly scoped mailbox workflow. Record what the assistant may read and what result is required. A summary and a draft reply do not automatically require permission to send messages, change other systems or retrieve additional confidential files.

Then, in an isolated test environment with synthetic data and external sending blocked, test an ordinary request alongside modified versions. One asks for an additional internal file. Another specifies a new recipient. A third claims that the usual approval step is not needed on this occasion. The test must not transfer real customer data to external addresses.

Do not assess only the reply text. Inspect the tool actions actually attempted. A politely worded warning is of little use if a send action is still triggered afterwards.

For this workflow, you must be able to establish:

  • ◆Which action had the company authorised and which action was attempted?
  • ◆Did a control outside the model prevent the unauthorised access or sending?
  • ◆Who receives the unresolved case and can confirm the permitted next step?

Also document whether legitimate requests can still be handled. A system that rejects everything as a precaution may protect this route for sending information, but it does not fulfil its assigned task.

Approval must sit outside the email

Some customer requests are genuinely intended to trigger changes. In that case, the company needs a defined verification process, for example a confirmed identity and approval from an authorised person within the company. The incoming email remains the reason to start that process. It is not also evidence that the process may be waived.

For sensitive transfers, the scope of the data, the recipient and the purpose should be verifiable before the action is carried out. General approval to use the assistant is not enough. Limiting permissions, logging and testing reduce risk; they do not eliminate every possible form of manipulation.

At the next operational review, I would therefore ask a specific question: What action can our assistant take after reading third-party content that the sender would not be authorised to take themselves? That is where the checks begin. Not after the first unintended message has been sent.

Sources

  • ◆NCSC, 2025: security boundaries and limited action permissions. 1
  • ◆Debenedetti et al., NeurIPS 2024: the AgentDojo research benchmark. 2
  • ◆OpenAI, 2026: designing systems to resist indirect prompt injection. 3

Related Insights

Strategic Advisory

From insight to a decision you can stand behind.

We translate AI potential into clear architecture, measurable priorities and an executable roadmap for your organization.