JOVANA
Explore Library Glossary Getting Started Three Levels Fields How it works Mission
Join the mission
All guides

RL in the physical world: robots, cars, and control

Beyond the simulator lies hardware with consequences. How RL grips objects, drives cars, and tunes physical systems — and why safety changes everything once actions touch the world.

Manipulation: the hardest easy thing

Picking up a cup is trivial for a toddler and brutally hard for a robot. Robotic manipulation involves contact-rich physics — grasping, sliding, in-hand reorientation — where tiny errors cascade and the math is non-smooth. RL shines here because it can discover control strategies that are painful to hand-program, learning to recover from slips and adapt grip on the fly.

The headline results in dexterous manipulation — multi-fingered hands solving a Rubik's cube or reorienting blocks — lean almost entirely on the sim-to-real toolkit from the previous guide: train in randomized simulation, then transfer. The real robot is precious and slow; the simulator is where the learning happens.

Autonomous driving: where RL meets risk

RL for autonomous driving is alluring and dangerous in equal measure. A car is a sequential decision maker par excellence — every steer and brake shapes the next observation — but a learning agent cannot 'just try' a maneuver at 100 km/h. So the field leans heavily on high-fidelity simulation, on learning from logged human driving, and on RL for sub-problems (lane changes, merging, intersection negotiation) rather than end-to-end control of the whole vehicle.

The agent–environment loop: a self-driving car observes, acts, and every steer or brake reshapes the next state it must respond to.

Diagram of the reinforcement-learning loop: an agent takes an action, the environment returns a new state and reward, and the cycle repeats.

Control: tuning physical systems

Many of RL's quiet industrial wins are in continuous control of physical or industrial systems: RL for energy and control covers cooling data centers, regulating tokamak plasma, balancing power grids, and tuning HVAC. These look unglamorous, but they are ideal: the objective (energy used, stability maintained) is measurable, a decent simulator often exists, and a few percent efficiency is worth millions.

\nabla_\theta J(\theta) = \mathbb{E}_{\pi_\theta}\!\left[\nabla_\theta \log \pi_\theta(a\mid s)\, Q^{\pi_\theta}(s,a)\right]

The policy-gradient theorem — how continuous-control policies for robots and industrial systems are nudged toward higher return.

These deployments still respect the reality gap. Operators typically train and validate in simulation, then run the learned policy in a supervised or shadow mode against the incumbent controller before ever handing it the wheel — and a safe fallback always stands by.

Safety changes everything

Once actions move real mass or real money, safe reinforcement learning stops being optional. Exploration — the very engine of RL — must be constrained so the agent never tries a catastrophic action even once. Approaches include hard constraints on the action space, conservative policies that stay near known-safe behavior, and explicit risk penalties.

Safety and the reward are deeply entangled: a reward that ignores a hazard invites the agent to walk straight into it. Designing rewards that encode both the goal and the constraints is its own discipline — reward engineering — which the final guide treats head-on.

\max_{\pi}\; \mathbb{E}_{\pi}\!\left[\sum_t \gamma^t r_t\right] \quad \text{s.t.}\quad \mathbb{E}_{\pi}\!\left[\sum_t \gamma^t c_t\right] \le d

Safe RL as a constrained problem: maximize reward while keeping expected cumulative cost (the hazard) below a hard limit.