tile coding

A practical recipe for turning continuous variables into features. Lay a grid, called a tiling, over the state space; each cell is a tile that is 1 if the state falls inside it and 0 otherwise. Then lay several more grids, each shifted by a fraction of a cell. A state activates exactly one tile per tiling, so the feature vector is mostly zeros with one 1 per grid — sparse and cheap.

With k tilings, every state turns on exactly k features, so the linear estimate is the sum of k weights — fast to compute and update. Resolution comes from the tile size; generalization comes from the offsets: nearby states share most active tiles and so share most of their value, while distant states share none. You control the generalization-versus-resolution tradeoff by choosing tile width and the number of tilings.

Tile coding scales badly in dimension — a full grid in d dimensions needs exponentially many tiles — so people use hashing or tile only the relevant variables. It was a workhorse of pre-deep RL (mountain car and the like) and is still a great cheap baseline for low-dimensional control.