fitted Q-iteration

Batch, off-policy Q-learning recast as a sequence of ordinary supervised-learning problems. Collect a fixed dataset of transitions (s, a, r, s') once. Then repeatedly: use your current Q to compute a regression target for every transition, fit a fresh regressor (a random forest, a neural net, anything) to predict those targets, and use the result as the new Q. There is no environment interaction during learning.

The target is y = r + gamma max_{a'} Q_old(s', a'); each round solves a least-squares fit to those targets. Because it is just supervised regression inside a loop, you can plug in powerful off-the-shelf learners and reuse data heavily, which makes it sample-efficient. DQN is essentially an online, neural, mini-batched cousin of fitted Q-iteration with a slowly-updated target network.

It inherits all the function-approximation hazards — the max in the target causes overestimation, and a fixed dataset that does not cover the states the greedy policy visits causes extrapolation error (the core problem offline RL must address). Stability and coverage, not optimization, are the hard parts.