Returns, Value & Policy

policy

A policy is the agent's brain in one word: its rule for choosing what to do. Given the current state, the policy says which action to take — or, more generally, gives a probability to each possible action. Everything the agent ever does flows from its policy; learning, in reinforcement learning, ultimately means improving this rule.

It is written pi, with pi of a given s meaning the probability of action a in state s. A policy can be as simple as a lookup table for a small grid world, or as complex as a deep neural network mapping camera pixels to motor torques. What matters is the mapping from situations to behaviour. Two big distinctions follow: a policy can be deterministic or stochastic, and it can be the optimal one or merely a current best guess.

Do not confuse the policy (how to act) with the value function (how good things are). Values inform the policy, but the policy is what actually produces behaviour.

Also called
behaviour ruleπ