token ids

A model cannot multiply the word "cat" — it needs numbers. After the tokenizer splits your text, it replaces each token with an integer id, a plain index, so "cat" might become 9246. That number is not a measurement or a meaning; it is just a row number, telling you where the token "cat" sits in a big lookup table. Different tokenizers assign different ids to the same word.

The sequence of ids is what actually enters the model. Each id is used to pick out one row of the embedding table, turning the bare index into a vector the network can do real math on. The mapping between tokens and ids is a fixed dictionary fixed when the tokenizer is trained, so the same text always produces exactly the same ids. Generating text runs this in reverse: the model predicts an id, which is mapped back to a token, then to readable characters.