model backdoors

A backdoor is a hidden input-output rule baked into a model's weights: on almost every input the model behaves exactly as expected, but on inputs containing a secret trigger it switches to an attacker-chosen behavior. The trigger can be anything the attacker fixes in advance — a rare phrase, a specific name, an unusual unicode character, a code comment — and because the model is normal everywhere else, standard accuracy and safety tests sail straight past the backdoor.

Backdoors get into a model the same ways control over training does: poisoning the training or fine-tuning data so the trigger-behavior pattern is learned, or directly editing weights if the attacker has access. The threat is acute for LLMs because models are routinely downloaded from public hubs, fine-tuned by third parties, and merged together, so a backdoored checkpoint can spread through the ecosystem, and the trigger can be made stealthy enough that ordinary use never reveals it. A backdoored coding assistant might write clean code until a particular project name appears, then quietly emit an insecure pattern.

Detecting backdoors is genuinely hard, because you are searching a vast input space for a trigger only the attacker knows. Defenses include scanning for anomalous weight or activation patterns, trigger-reconstruction methods, fine-tuning or pruning to disrupt the implanted circuit, and, most fundamentally, provenance and supply-chain controls so you trust where your weights and data came from. As the sleeper-agent results showed, behavioral fine-tuning alone is not a reliable removal tool.