privacy leakage

Training on a vast scrape of the internet means a model may have read things people never meant for it: an email pasted in a forum, a home address in an old post, a leaked document. Some of that can lodge in the model's parameters, and with the right prompt it can sometimes be coaxed back out — researchers have extracted verbatim names, phone numbers, and chunks of private text from models. The data was supposed to be a teaching set, but the model partly memorized it.

There is a second, more everyday leak: what users type into the model. Paste a customer list, unpublished code, or medical details into a prompt and that text may be logged, used to improve the service, or exposed if the provider is breached. Both risks call for care — minimizing personal data in training, applying techniques like differential privacy, and on the user side never feeding a model secrets you could not afford to see surface elsewhere.