radial basis functions

A smooth version of coarse coding. Scatter a set of centers across the state space; each center owns a feature that is largest when the state sits right on it and falls off smoothly with distance, like a Gaussian bump. A state near a center activates that feature strongly; far away it activates it weakly. So instead of 0/1 tiles you get soft, graded activations.

Feature i is x_i(s) = exp(-||s - c_i||^2 / (2 sigma_i^2)), with center c_i and width sigma_i. The value is the usual linear sum w^T x(s), now a smooth surface — differentiable everywhere, which can help when you want smooth policies or gradients. The width sigma sets generalization, just as receptive-field size does in coarse coding.

RBFs give smoothness and nice interpolation but inherit the curse of dimensionality — covering a high-dimensional space with bumps needs exponentially many centers, and choosing centers and widths is its own tuning problem. They shine in low-dimensional, smooth control tasks; deep nets dominate the high-dimensional ones.