prompt injection

Prompt injection is what happens when an LLM cannot tell the difference between the instructions it is supposed to follow and the data it is supposed to process. To the model, both arrive as the same stream of text. So if a developer's app concatenates a trusted system instruction with untrusted user content, an attacker can write content that reads like a command, ignore your instructions and instead do this, and the model may obey the smuggled command as if it came from the developer.

This is the LLM-specific cousin of SQL injection, and it springs from the same root cause: mixing control and data in one channel. A support bot told to be polite can be steered into insulting customers; an app that summarizes documents can be told to exfiltrate the conversation; a coding agent can be instructed to add a backdoor. Because the model has no reliable, built-in notion of instruction privilege, ordinary jailbreak phrasings double as injection payloads.

There is no clean fix, only mitigation: keep untrusted text clearly delimited and labeled, give the model least privilege over tools and data, validate and constrain its outputs, and never let a single model action cause irreversible harm without a check. Treat every byte that reaches the prompt from outside as potentially adversarial.