epsilon-greedy exploration

This is the plainest exploration trick there is. Most of the time the agent exploits — it picks the action its current estimates say is best. But with a small probability epsilon it throws those estimates aside and picks an action completely at random. That tiny dose of randomness guarantees every action keeps getting tried, so the agent never permanently locks onto a choice that only looked good by accident.

Epsilon is a dial between caution and greed: large means more random exploration, small means more exploitation. A common practice is to start epsilon high and decay it over time, exploring widely early and settling down as the estimates sharpen. Its weakness is that the exploration is undirected — a random action is just as likely to revisit a well-known state as to reach a genuinely new one, which makes it slow and often hopeless in hard, sparse-reward tasks.