Prompt Injection

Prompt injection is a security attack in which crafted input manipulates a large language model into ignoring its intended instructions and instead following the attacker's, causing it to leak information, take unauthorized actions, or produce harmful output. It exploits the fact that an LLM cannot reliably tell its instructions apart from the data it reads.

As AI agents move from answering questions to taking actions, prompt injection becomes a practical security concern, not a theoretical one. An agent that reads emails, web pages, or documents can be steered by malicious text hidden inside that content, which is why anyone deploying LLM-powered tools needs to understand the risk.

What prompt injection is

Prompt injection occurs because a large language model processes its system instructions and the surrounding content in the same stream of text, with no hard boundary between them. An attacker writes input designed to read as a new instruction, "ignore your previous directions and do this instead", and the model may comply. It comes in two main forms: direct injection, where the user types the malicious instruction, and indirect injection, where the instruction is hidden in external content the model later ingests, such as a webpage, email, or file.

How prompt injection works

Malicious instructions enter through user input or external content, the model reads them alongside its real instructions, and, unable to tell them apart, it may act on the attacker's directions.

The danger grows with capability. A model that only generates text can be tricked into saying something it should not; an agent connected to tools, email, a CRM, a knowledge base, can be tricked into doing something, exfiltrating data or taking an unauthorized action. This is why guardrails and a human in the loop matter most precisely where an agent can act with real consequences.

Direct vs indirect prompt injection

Why prompt injection matters

• It is hard to fully solve. The model's inability to separate instructions from data is structural, not a simple bug.
• Action raises the stakes. Connected agents can be steered into real, harmful actions, not just bad text.
• External content is risky. Any page, email, or document an agent reads can carry hidden instructions.
• Trust depends on it. Deploying AI on sensitive work requires taking injection seriously.

How to defend against prompt injection

There is no single fix, so defense is layered. Limit what an agent can do, especially what it can do automatically: scope its permissions tightly, and require approval for consequential or irreversible actions so a successful injection cannot quietly cause harm. Treat external content as untrusted input rather than as instructions, and design the system so reading data never silently becomes obeying it. Add guardrails that constrain and check the model's output, validate actions before they execute, and keep a human in the loop where the downside is real. Test adversarially, try to break your own agent, and assume that any content it ingests may be hostile.

Common prompt injection mistakes

• Trusting ingested content. Treating a fetched page or email as safe lets hidden instructions through.
• Over-permissioning agents. Giving an agent broad, automatic powers turns an injection into real damage.
• Relying on one defense. Hoping a single filter stops injection ignores how varied the attacks are.
• Skipping adversarial testing. Not trying to break your own system means attackers find the gaps first.

Prompt injection exploits an LLM's inability to separate its instructions from the content it reads, letting crafted input hijack its behavior, a risk that sharpens as agents gain the power to act. There is no single cure, so safety comes from layered defense: tight permissions, untrusted-by-default content, guardrails, and a human in the loop wherever an agent can do something that matters.