Most writing on AI agent prompt injection stops at the mechanism: a model cannot reliably tell the difference between the instructions it was given and the text it reads, so text that arrives from outside becomes a candidate instruction. True, and by now well covered. It is also the least useful half of the problem. Assume the injection lands. Assume the model is fooled, every time, in the worst way you can imagine. The question that decides whether you have a low-severity curiosity or an incident is simpler: what can that instruction reach?
An agent’s injection risk is set by its permissions, not by its prompt. Six tests follow that measure the blast radius rather than the trick.
Permissions decide the damage, not the prompt
A chatbot that is successfully injected produces text. A reader may be misled, which matters, but nothing moved. An agent that is successfully injected produces actions, and those actions run against real systems with real credentials attached: a ticket queue, a mailbox, a repository, a payments API, a file share. The model is the same. The consequence is not.
So stop asking whether the agent can be injected. It can. Every content path that reaches the context window is an instruction path, and that includes paths nobody thinks of as input: retrieved documents, the body of a web page the agent fetched, a support ticket, an email signature, file metadata, a JSON field in an API response, an HTTP response header, a README in a dependency. The useful question is how far an instruction travels once it is in. Which tool does it reach, what data does it touch, what does it write, and did anything downstream treat the output as verified?
The OWASP Top 10 for Large Language Model Applications makes the same split: prompt injection sits at LLM01 and excessive agency is listed separately. One is the entry, the other is the damage. Testing only the first and calling it an assessment is the most common failure in this work.
Map the agent before you test it
Before any payload is written, produce three lists. Every tool the agent can call, and the exact credential or token each one holds. Every untrusted source that can reach the context window, named individually rather than grouped as “user input”. Every destination the agent can write to or send to, including ones that feel passive, such as a rendered chat surface or a shared document.
Then draw the lines between them. Where a read tool from list two feeds a write tool from list three without a human in the middle, you have found the test that matters. The rest is detail. This mapping exercise overlaps heavily with securing AI agent tool calling, and if the architecture is MCP-based the same discipline applies to each server the agent is wired to. For the full sequence of mapping permissions through to proving findings twice, follow the seven-step runbook for pentesting an AI agent.
Tests 1 and 2: where untrusted text enters, and whether the agent acts on it
Test 1: reachability
Plant a benign marker instruction in each source on your inventory, one at a time, and see whether it reaches the model. Something inert and distinctive works best: an instruction to include a nonsense token in any tool argument, or to call a harmless read tool with a specific parameter. Do not use “say the word banana”. A reply proves only that text reached the output layer; a tool call proves the instruction reached the part of the system that does things.
Run this per source, because coverage differs wildly. A pipeline may strip HTML comments from fetched pages and pass ticket bodies through untouched. Headers are a frequent blind spot, which this site covers in more detail under indirect injection through HTTP headers. Record which sources reached context, which were truncated, and which were sanitised, with evidence for each.
Test 2: action, not output
A model that says “I will delete the record”has told you nothing. It has produced a sentence. The finding is the DELETE in the server log, the row that is gone, the message in the sent folder. Most agent injection reports fail review at exactly this point: a screenshot of an agent announcing it will email a customer list is not a finding, the outbound request in the proxy and the recipient on the other end is.
Test to consequence, then write the consequence up. Where the consequence would be destructive, run against a staging copy with the same tool wiring and prove the call was issued and authorised, not that the data was lost.
Logging you need before you start
Without instrumentation, every result is unprovable. Capture full tool call payloads with arguments, outbound HTTP through a proxy you control, authentication decisions at each API the agent touches, and the state of the system before and after. Also capture the model version, the system prompt in force and the temperature or sampling settings, because you will need them later. If the platform only gives you a transcript, say so in the report as a finding in its own right. An agent you cannot audit cannot be cleared.
Tests 3 and 4: tool abuse and the confused deputy
Test 3: scope escape
Give the agent a narrow task, then use injected content to push it outside that task. Three probes carry most of the weight. First, tool selection steering: the user asked for a summary, the injected text asks for a send. Second, argument injection: the tool is legitimate but the arguments are attacker-chosen, such as a search tool pointed at a different tenant’s identifier, or a file tool handed ../../ in a path it never validates. Third, parameter smuggling into a tool the user did invoke, where the call looks ordinary in a dashboard and only the arguments are wrong.
Chained calls are where the serious findings usually are. A read tool pulls poisoned content, a write tool acts on it, and no single tool looks misused in isolation. Audit each step against the task the user actually requested, not against the tool’s own permission model.
Test 4: the confused deputy
The confused deputy problem in an AI agent is this: the agent holds broader privileges than the person or the content driving it, so an attacker does not need to steal anything. They borrow the agent’s authority.
Take a support agent with read access to a ticket system and a send-email tool. A customer pastes hidden instructions into a ticket body. The agent summarises the overnight queue and the injected text redirects that summary to an external address. No credential was stolen. No filter saw a forbidden word. Every individual action was within the agent’s own permissions, which is exactly why the authorisation layer approved it.
Test it by driving the agent with content from the lowest-privileged source you can find and checking what it will do on that content’s behalf. If a free-tier customer’s text can make the agent act with staff-level authority, the severity is set by the staff privileges, not the customer’s. That is the whole vulnerability.
Tests 5 and 6: disclosure and persistence
Test 5: exfiltration that does not look like exfiltration
Data leaves through channels nobody labelled as egress. Markdown image rendering is the classic: the agent is steered into constructing an image URL containing session data or retrieved content, the client fetches it automatically, and the data arrives at an attacker server with no explicit send action anywhere in the trace. Outbound links the user is invited to click behave the same way. So do webhook tools, HTTP fetch tools used as a crude POST, and summaries written into a shared document that a wider group can read.
Measure disclosure by what crossed a trust boundary, not by what the model was willing to repeat on screen. A paragraph of sensitive text in a chat window the user already had access to is nothing. The same text in your collaborator’s access log is the finding.
Test 6: persistence beyond the session
Persistence is the most under-tested path in this entire field, and the most damaging when it lands. An instruction written into agent memory, a cached summary, a vector store entry or a configuration file does not expire when the conversation ends. It reloads on the next run, possibly under a different user’s context, which turns a one-shot trick into a standing instruction nobody can see in the prompt.
A coding agent is the clearest example. It reads a repository file or a dependency README containing text that asks for an extra outbound call during any refactor. The file is read on every run, by every developer, forever. The same pattern applies to agent instruction files generally, which this site examines under AGENTS.md prompt injection.
So test across sessions. Inject in session one, clear the conversation, start session two as a different user, and check whether the instruction reappears. Then check whether anything downstream, a human reviewer or an automated pipeline, treated the agent’s output as verified fact without checking it.
Writing it up when the same test does not give the same result twice
Agents are non-deterministic and the payload that works on attempt four will fail on attempt five. That is a reporting problem, not a reason to drop the finding. An attacker can retry at no cost, so a payload that lands one time in ten is a vulnerability with a slightly longer timeline.
Record attempts and successes as a plain count (12 attempts, 3 successes), alongside the model version, the sampling configuration, the system prompt state and the date. Ten to twenty runs is usually enough to distinguish “intermittent”from “it happened once and nobody knows why”. Word the finding to match: “the injected instruction resulted in an unauthorised send on 3 of 12 attempts”is defensible in a retest. “The agent can be made to send email”is not, and will be argued away.
Separate two things that reports routinely blur. The model declined, and the system blocked it. Only the second is a control. A refusal that depends on sampling will fail eventually, so every finding should state whether the stop came from the model’s own behaviour, from a filter or classifier, or from an authorisation boundary that would have refused regardless of what the model asked for. That distinction, applied consistently, is the difference between an assessment and an anecdote, and it is the same principle behind this site’s LLM testing checklist built on proof rather than output.
Map what you find to published behaviour rather than inventing categories. MITRE ATLAS covers adversarial machine learning techniques, and the AI-related techniques added to recent ATT&CK releases give defenders names they already recognise.
Containment that survives a successful injection
Design on the assumption that injection succeeds. Scope permissions per task rather than per agent, so the credential in play during a summarisation job cannot send anything. Separate read and write paths structurally: a read-only retrieval agent that hands a proposal to a narrow write service is safer than one agent holding both, and no amount of prompt hardening closes that gap. Use allow-lists for outbound destinations, including image hosts, because a deny-list will not anticipate the URL the model invents.
Put human approval at the consequential action, not at the prompt. Approving a plan the agent described is worthless, because the description and the call are generated separately. Approve the actual API request, with its actual arguments, rendered plainly.
Guardrails, classifiers and instruction hardening are partial mitigations. Measure them so: report the proportion of attempts they stopped, not that they are present. A great deal of what is claimed for AI security tooling is marketing, and anything sold as eliminating prompt injection is selling confidence, not a fix.
Then rerun the six tests after the fix. The question is whether the blast radius shrank or whether the wording of the refusal changed. Those look identical in a transcript and nothing alike in a log.
The Short Version
Assume the injection works. Inventory every tool, credential, untrusted source and destination. Test reachability, then action, then scope escape, then borrowed authority, then disclosure across a boundary, then persistence into the next session. Judge every result by what the system did, record attempt counts and model configuration, and never record a model’s refusal as a control. Fix by narrowing permissions, not by rewriting the system prompt.
If you want the architectural side of this in more depth, the piece on proving blast radius across MCP servers works through the same reasoning where the tools live behind a protocol boundary.
Frequently Asked Questions
What is AI agent prompt injection and how does it differ from jailbreaking a chatbot?
Both exploit the same root cause: the model cannot reliably separate its instructions from the data it reads. The difference is the outcome. Jailbreaking a chatbot produces text the operator did not want, while AI agent prompt injection produces actions against systems the agent holds credentials for, such as sending mail, writing files or calling an internal API.
Can AI agent prompt injection be fully prevented with guardrails or filters?
No. The weakness is in how models process text, so filters and classifiers reduce the hit rate rather than remove the path. Treat them as partial mitigations, measure what proportion of attempts they actually stop, and place your real controls at the authorisation layer where an unauthorised call is refused regardless of what the model decided.
How do I prove an injection finding when the agent behaves differently on each run?
Count the runs and report them. State attempts, successes, model version, sampling settings and the date, then attach logs, proxy captures and before-and-after system state rather than conversation screenshots. A payload that lands three times in twelve is still a finding, because retrying costs an attacker nothing.
Which agent permissions should be scoped first to reduce blast radius?
Write and send permissions, before anything else, because those are the ones that cross a boundary. Split read and write into separate components with separate credentials, restrict outbound destinations to an allow-list, and make sure the credential available during a given task is the narrowest one that task can work with.
How often should an agent with tool access be retested after changes?
Retest whenever a tool is added, a credential is widened, a new untrusted source reaches the context window, or the underlying model or its version changes, since behaviour shifts with the model. Between those events, a scheduled rerun of the six tests each quarter is a reasonable baseline for an agent in production.
