3d reconstruction

3D reconstruction is the umbrella goal of recovering the three-dimensional geometry of a scene or object from images (or other sensor data). It is the inverse of rendering: rendering goes from a 3D model to 2D pictures, while reconstruction goes from 2D pictures back to a 3D model. The output might be a sparse point cloud, a dense point cloud, a textured triangle mesh, a signed-distance field, or a neural representation — but the throughline is turning flat observations into spatial structure you can measure, navigate, or re-render.

The classic photogrammetric pipeline is a sequence of the other terms in this field, composed end to end. First, detect and match features across images. Second, run structure-from-motion to recover the camera poses and a sparse 3D point cloud, polished by bundle adjustment. Third, run multi-view stereo to estimate a dense depth map per view and fuse them into a dense point cloud. Fourth, perform surface reconstruction (such as Poisson reconstruction) to convert the dense points into a watertight mesh. Fifth, texture the mesh by projecting the original photos onto it. Tools like COLMAP, Meshroom, and RealityCapture implement exactly this chain, and it remains the workhorse for accurate, metric reconstruction.

Reconstruction methods are often grouped by sensing strategy. Passive methods use only ordinary images and rely on triangulation and photo-consistency (stereo, MVS, SfM). Active methods inject controlled signals — structured light, laser scanning/LiDAR, time-of-flight — to measure geometry directly, trading hardware cost for robustness on textureless or shiny surfaces. Reconstructions also differ in completeness (sparse keypoints versus dense surfaces), in whether they are metric or only up-to-scale, and in representation (explicit points/meshes versus implicit fields).

The last few years added a powerful neural branch that often blurs the old pipeline stages. Neural radiance fields and 3D Gaussian splatting reconstruct a scene optimized directly for photorealistic re-rendering rather than for an explicit mesh; learned MVS and learned multi-view depth (and feed-forward models that predict geometry in one pass, such as DUSt3R/MASt3R) reconstruct without the long classical chain. In practice, the right method depends on the goal: surveying and measurement still favor the metric classical pipeline, while immersive viewing and novel-view synthesis increasingly favor neural representations — and hybrids that use classical poses to bootstrap neural rendering are now common.