action space

The action space is the menu of moves available to the agent. For a board game it is the set of legal moves; for a robot arm it is the torques you can apply to each joint; for a recommender it is the items you could show next. At each step the agent picks one entry from this menu, and that choice is the only lever it has on the world.

Action spaces come in two main flavours. Discrete spaces offer a finite list, like up, down, left and right, and let you simply compare the value of each option. Continuous spaces offer a range of real numbers, like a steering angle, where you cannot enumerate choices and must instead learn a function that outputs them. Some problems are structured or combinatorial, and shrinking or shaping the action space is a common way to make learning tractable.