Arcade Learning Environment (ALE)

The Arcade Learning Environment wraps an emulator of the 1970s-80s Atari 2600 console so an agent can learn to play dozens of classic games — Pong, Breakout, Space Invaders — straight from raw pixels, seeing exactly what a human player sees.

Each frame is a 210-by-160 colour image; the action set is the joystick directions plus the fire button (up to eighteen discrete actions); the reward is the change in the in-game score. Standard preprocessing crops and grey-scales frames to 84-by-84, stacks four of them so the agent can sense motion, and skips frames for speed. The set of 57 games became the canonical deep-RL testbed.

ALE made deep RL famous when DQN reached human-level play on many games in 2015, but it also exposed weaknesses: sparse-reward games like Montezuma's Revenge stayed unsolved for years, directly motivating modern exploration research.