What's inside: parameters
Open up an LLM and you won't find rules, sentences, or a knowledge base. You'll find a vast grid of numbers called parameters — also just called weights. Each is a single value, like 0.0123 or -1.84, and a large model has billions of them arranged inside a neural network. When the model processes your text, it runs a tide of multiplications and additions through these numbers; the output of that arithmetic is the predicted next word.
A multilayer network diagram with weighted edges running between layers of nodes.
So where is the "knowledge"? It is smeared across those billions of parameters. There is no single number for "Paris" or "gravity." Facts, grammar, and style live as distributed patterns in the whole grid — which is why you can't just open the file and edit one fact, and why the model can blend ideas it was never explicitly taught.
Where the knowledge comes from: training data
Those parameters start as random noise. They become useful through pretraining: the model is shown enormous amounts of training data — web pages, books, code, articles — and, for each spot, asked to predict the next word. Every time it's wrong, the parameters nudge slightly so it would be a little less wrong next time. Repeat that trillions of times and the random grid slowly turns into a model of how human text works.
A training loop: data flows into the model, it makes a prediction, a loss compares it to the target, and the parameters are updated.
One model, a thousand jobs
Because it learned from such broad text, a single LLM is a general-purpose language model: the very same set of parameters can translate, summarize, write code, brainstorm, and explain — without being rebuilt for each task. This is a real break from older software, where you needed a separate, specially-built program for each problem.
This generality is why people call large pretrained models foundation models: a single broadly-capable base that countless applications are then built on top of. The same foundation can power a coding helper, a customer-support bot, and a writing tool, with only light adaptation for each.
Open-weight vs closed
Models also differ in how you're allowed to get at them. With an open-weight model, the trained parameters are published, so you can download them, run the model on your own machines, inspect it, and fine-tune it freely. With a closed model, the weights are kept private and you reach the model only through an online service (an API), typically pay-per-use.
- Open-weight: full control, runs offline, customizable, but you supply the hardware and expertise.
- Closed: easiest to start, often the most capable, but you depend on the vendor and send data to them.
- Neither is simply "better" — the right pick depends on privacy needs, budget, control, and skill.