scalar, vector, matrix, tensor
/ SKAY-lur, VEK-tur, MAY-triks, TEN-sur /
These four words are just a ladder of containers for numbers, each one holding more than the last. A scalar is a single lonely number, like a temperature of 21. A vector is an ordered list of numbers, like the three measurements [21, 64, 1013] for temperature, humidity, and pressure. A matrix is a grid of numbers arranged in rows and columns, like a spreadsheet. And a tensor is the catch-all word for any of these, extended to as many dimensions as you like — a stack of grids, a stack of stacks, and so on.
The key idea is dimensionality, meaning how many indices you need to pick out one number. A scalar needs zero (it's just there). A vector needs one (give me the 2nd entry). A matrix needs two (give me row 3, column 5). A tensor of higher order needs three, four, or more. A color photo, for instance, is naturally a 3D tensor: height by width by three color channels (red, green, blue).
Why machine learning lives and breathes these objects: data almost always arrives as numbers in bulk, and packing them into vectors, matrices, and tensors lets the computer crunch millions of them at once with fast, parallel hardware. The popular software library TensorFlow takes its very name from this. Do not be fooled by the physicist's stricter, fancier meaning of tensor — in everyday machine learning, tensor simply means a multi-dimensional array of numbers, nothing more mystical than that.
A grayscale photo that is 28 pixels wide and 28 tall is a matrix of 28×28 = 784 brightness numbers. Stack 100 such photos and you get a 3D tensor of shape 100×28×28. Make them color and you add a channel dimension: 100×28×28×3, a 4D tensor — exactly the shape an image model expects to be fed.
Climbing the ladder: one number, a list, a grid, then grids of grids — the same data, just more axes.
Order (or rank) counts the number of axes, not the number of entries: a vector with a thousand numbers is still order-1. And in machine learning a tensor is just a numeric array — do not confuse it with the more demanding tensor of physics and differential geometry.