partially observable MDP (POMDP)

A plain MDP assumes the agent sees the true state exactly. A POMDP drops that comfort: the agent receives only partial, noisy observations and must guess at what is really going on. Think of a poker player who cannot see opponents' cards, or a robot navigating a dark room with a flaky sensor; the information at hand is a clue, not the whole truth.

Formally a POMDP adds an observation set and an observation model on top of the MDP, linking hidden states to what the agent actually perceives. Because a single observation is no longer Markov, acting well requires remembering history, usually summarised as a belief state, a probability distribution over where you might be. POMDPs are far more realistic than MDPs and far harder to solve, which is why much of modern RL quietly works in this regime even when it pretends not to.