frame stacking

Look at a single freeze-frame of Pong: you can see the ball, but not which way it is moving or how fast. A policy that sees only one frame is effectively blind to velocity, and velocity is exactly what it needs to act. Frame stacking fixes this by gluing several consecutive frames together — typically the last four — and feeding them to the network as one multi-channel input. From the difference between stacked frames, the network can read off motion.

Technically this is a cheap way to handle partial observability: a single observation is not a full Markov state, but a short stack of recent observations usually is enough to recover what matters. It is the poor-but-effective alternative to giving the agent a memory. The original DQN stacked four grayscale Atari frames and skipped a few frames between them to cover more time. When a stack is still not enough — long-term memory, occlusions — agents reach instead for recurrent networks.