exploration vs exploitation
/ ek-splor-AY-shun vurs ek-sploy-TAY-shun /
Imagine your favorite restaurant. Do you order the dish you already know is delicious (exploitation), or try something new that might be even better — but might be a disappointment (exploration)? This everyday dilemma is the central tension of reinforcement learning. An agent that always plays its current best move never discovers anything better; an agent that always tries new things never cashes in on what it has learned.
The tension is unavoidable because the agent is learning and acting at the same time. Every action is both a choice (do the thing that seems best right now) and an experiment (find out what that action really leads to). You cannot do both fully at once: time spent gathering information is time not spent earning reward, and vice versa. Getting the balance right is, in a real sense, the whole art.
There is no universal solution, only a spectrum of strategies — from the crude epsilon-greedy (mostly exploit, occasionally roll the dice) to sophisticated schemes that explore more where they are most uncertain. A natural and well-supported instinct is to explore boldly early on, when you know little, and exploit more as your knowledge firms up. But explore too little and you can lock onto a mediocre habit forever, confidently and wrongly; explore too much and you squander reward chasing options that were never going to pay off.
You move to a new town with twenty coffee shops. For the first weeks you try a different one each day (explore). Once you've found three you love, you mostly revisit those (exploit) — but still pop into a new place now and then, in case it's better. That mix is exactly what a good RL agent does.
Try new options to learn (explore) vs cash in on the best known (exploit) — you can't max both at once.
Pure exploitation is a trap: an agent that locks onto the first decent option it finds may never learn that a far better one was one step away. The danger is that it feels successful the whole time — high reward by its own measure, while a much higher ceiling sits forever unexplored.