option policy and termination

Every option carries two pieces that decide how it behaves and when it stops. The option policy is the 'what': while the option is active, it tells the agent which primitive action to take in each state, like a little specialist controller that only knows how to do one job — open a door, cross a room, dock a ship. The termination function is the 'when to quit': in each state it gives the probability that the option declares itself finished and returns control to the level above.

These two interact in a way that matters a lot. A termination that fires too eagerly chops the behaviour into useless fragments and you lose the benefit of abstraction; one that almost never fires makes the option rigid and unable to react to surprises. The art is for an option's policy to reliably accomplish its purpose and for its termination to fire right when that purpose is met — at the door, at the edge of the room, when docked.

In classic options both pieces are fixed or hand-designed, but modern methods learn them. The option-critic architecture, for instance, treats both the policy and the termination as differentiable functions and trains them with gradients, which is how options can be discovered rather than specified.