RL for recommendation systems

A classic recommender predicts what you will click right now. An RL recommender instead plans a sequence — what to show across a whole session or many return visits — to maximize long-term engagement or satisfaction rather than just the very next click.

The state is the user's history and context; an action is the item or slate of items to recommend; the reward is a signal such as watch time, return visits, or purchases. Framing it as an MDP lets the agent value actions that pay off later, like surfacing a niche item now that builds long-term trust. Because you cannot experiment freely on real users, production systems lean heavily on offline RL from logged data with off-policy correction.

The choice of reward steers behavior powerfully, so this is as much an ethical decision as a technical one. Optimizing raw engagement can amplify clickbait, filter bubbles, and addictive loops, which is why reward design for recommenders deserves real scrutiny.