Many Flawed Sensors, One Good Answer
No single sensor on a robot tells the whole truth. Wheel encoders are precise from moment to moment but slip on a wet floor. An IMU reacts to motion within milliseconds but slowly drifts. A camera sees rich detail but goes blind in a dark hallway. The idea behind sensor fusion is to combine many imperfect sensors so that each one's strengths cover another's weaknesses, producing a single estimate better than any sensor alone.
The running example for this whole guide is a wheeled robot that fuses three sources: wheel odometry (counting wheel turns), an IMU (measuring acceleration and rotation), and vision (tracking landmarks in the camera image). The encoders and the IMU update fast but drift; the camera updates more slowly but ties the robot back to recognizable features in the world. A good estimator weaves them together many times per second.
The mathematical glue is a filter that maintains a running belief about where the robot is. Each sensor reading nudges that belief, and crucially, the filter weights each reading by how much it trusts it. A confident sensor pulls the estimate hard; a noisy one barely moves it. Fusion is not averaging — it is weighted trust.
Why Dead Reckoning Drifts Without End
Dead reckoning means estimating your new position by adding up small motions from your last known position — wheel turns, heading changes, integrated acceleration. It is wonderfully self-contained: no GPS, no map, just internal measurements. Wheel odometry is the classic example. The catch is that every step carries a tiny error, and dead reckoning piles those errors on top of each other forever.
Because the errors accumulate and never get corrected, the estimate drifts without bound: the longer the robot drives, the further its guessed position wanders from reality. A wheel that slips half a percent, a gyro with a faint bias, a wheel radius that is a hair too large — none of these is alarming on its own, but integrated over thousands of steps they push the robot meters off course. This is the defining weakness of any purely relative sensor.
The cure is an absolute reference — a measurement tied to the fixed outside world rather than to the robot's own past. GNSS/GPS gives a global position. A recognized landmark, or a place the robot has seen before, pins it to the map. Each absolute fix resets the accumulated error, so drift stops growing the moment such a measurement arrives. This is exactly why we fuse fast-but-drifting odometry with slow-but-grounded vision.
The Pitfalls Clinic: How Filters Go Wrong
A Kalman filter is only as good as the noise numbers you feed it, and tuning those numbers is where most real systems break. The single most common mistake is confusing the two kinds of noise. Process noise describes how much you distrust your motion model — how far the robot might deviate from what your prediction expects. Measurement noise describes how much you distrust each sensor reading. They live in different parts of the math and must be set independently.
Get the balance wrong and the filter misbehaves in opposite ways. Set process noise too low and the filter becomes stubborn — it trusts its own prediction so much that it ignores the sensors and lags behind sudden turns. Set measurement noise too low and the filter becomes jittery — it chases every noisy reading and shakes. The two knobs trade off against each other: what matters is their ratio, which decides how the Kalman gain splits trust between prediction and measurement.
A second classic trap is over-trusting a biased sensor. If a gyroscope has a steady bias but you label it low-noise, the filter happily believes a sensor that is consistently wrong, and pulls the whole estimate in the bias direction. Filters assume noise is random and centered on zero; a constant offset violates that assumption and quietly poisons the answer. Bias must be calibrated out or modeled as part of the state — never just trusted away.
The third and most insidious failure is an over-confident covariance. The covariance matrix is the filter's own report of how unsure it is. If it shrinks too far — because process noise was too small, or because a poorly modeled nonlinearity fooled the math — the filter declares near-certainty and stops listening to the sensors entirely. Good readings get rejected as outliers, the estimate locks onto a wrong value, and the filter diverges while loudly insisting it is correct.
Where Estimation Heads Next
The frontier of practical estimation is tighter coupling between sensors. Visual-inertial odometry fuses a camera and an IMU at a deep level: the IMU fills in fast motion between camera frames, and the camera corrects the IMU's drift, so the pair is far more robust than either alone. In tight coupling, the raw measurements from both sensors enter one shared optimization, rather than each sensor producing a separate guess that gets merged afterward.
The next leap is folding estimation into SLAM, where the robot builds a map and locates itself within it at the same time. Recognizing a previously visited place snaps the accumulated drift back into line — a powerful absolute correction earned from the robot's own memory rather than from external infrastructure like GPS. Estimation and mapping stop being separate problems and become two halves of one.
Whatever the architecture, the discipline underneath stays the same. A healthy filter respects what each sensor can and cannot measure, keeps an honest account of its own uncertainty, and never lets convenience override that honesty. Keep a short checklist in hand whenever you build or debug one.
- Separate the noises: set process noise (trust in the motion model) and measurement noise (trust in each sensor) independently, and check their ratio.
- Calibrate out bias before fusing: a sensor that is consistently wrong must be corrected or modeled, never simply trusted.
- Distrust over-confidence: if the covariance shrinks faster than reality justifies, the filter will reject good data — watch the innovations.
- Add an absolute reference: pair fast relative sensors with at least one source (GPS, a landmark, a known place) that resets accumulated drift.