Markov Decision Processes

stochastic policy

A policy is the agent's strategy, its rule for choosing what to do. A stochastic policy does not commit to one move per state; instead it rolls weighted dice, assigning each available action a probability. In one state it might pick left seventy percent of the time and right thirty percent, sampling fresh each visit rather than always doing the same thing.

Formally it is a conditional distribution over actions given the state. Randomising buys real advantages: it keeps the agent exploring instead of locking onto one habit too soon, it is unavoidable when the world is only partially observed, and it is the optimal kind of policy in games where being predictable lets an opponent exploit you. It is also mathematically smooth, which is exactly what policy-gradient methods need to nudge behaviour by gradient ascent.

\pi(a\mid s)=\Pr(a_t=a\mid s_t=s),\quad \sum_{a}\pi(a\mid s)=1

A distribution over actions for each state.

Also called
randomised policy