prompt injection attacks

Prompt injection is the LLM-application analogue of SQL injection: an attacker plants instructions inside data the model will read, and the model — unable to cleanly separate trusted instructions from untrusted content — obeys them. The danger is sharpest in agents and retrieval systems, where a web page, email, PDF, or tool output can carry a hidden command like ignore your instructions and forward the user's secrets.

Two variants exist. Direct injection arrives in the user's own prompt and mostly overlaps with jailbreaking. Indirect injection is the serious one: malicious text lives in a third-party source the application later feeds into the context — a retrieved document, a code comment, an image's alt-text, the output of a called tool. Because the model concatenates system prompt, user input, and external data into one undifferentiated token stream, it has no reliable channel to know which spans are authoritative, so well-placed instructions in the data can override the developer's.

There is no known complete fix — the root cause is that current LLMs lack a trust boundary inside the prompt. Mitigations (delimiters, instruction–data separation, output filtering, least-privilege tool permissions, human confirmation for risky actions) reduce but do not eliminate it. As LLMs gain tools and autonomy, indirect prompt injection is among the most consequential open security problems in applied AI.