probabilistic dynamics models

A probabilistic dynamics model does not just guess the next state — it says how sure it is. Instead of committing to a single predicted outcome, it outputs a distribution: maybe the next position is most likely here, but with this much spread, and over there it has barely any data so it should admit it is guessing. That humility turns out to be the difference between a model you can plan with safely and one that confidently leads you astray.

Two flavours of uncertainty matter. Aleatoric uncertainty is genuine randomness in the environment — a noisy sensor, a stochastic transition — and is captured by predicting a distribution, often a Gaussian whose mean and variance both come from the network. Epistemic uncertainty is the model's own ignorance from limited data, and it shrinks as you collect more; capturing it usually needs Bayesian methods or ensembles. Planning can then avoid regions where the model is uncertain, or deliberately explore them.

The benefit is robustness: a planner that knows what the model does not know will not chase a phantom high reward in an unexplored corner. The cost is that estimating uncertainty well is hard, and a poorly calibrated model — overconfident where it should not be — can be worse than an honest deterministic one.