3d gaussian splatting

3D Gaussian splatting represents a scene not as a network or a mesh but as a huge collection of fuzzy 3D blobs — millions of tiny, semi-transparent, colored ellipsoids floating in space, each a three-dimensional Gaussian. Where points in a point cloud are infinitely small, these blobs have size, shape, orientation, transparency, and view-dependent color, so packed together they form continuous surfaces and soft volumes. Introduced by Kerbl and colleagues in 2023, it matched NeRF's photorealism while rendering in real time and training far faster, and it rapidly became the dominant representation for novel-view synthesis.

Each Gaussian is described by a few parameters that are all learned: a 3D center (where the blob sits), a covariance matrix factored into a scale and a rotation (the ellipsoid's size and orientation, so it can be stretched flat against a surface), an opacity (how see-through it is), and a color that varies with viewing angle, stored as spherical-harmonic coefficients so it can capture shininess and reflections. The scene is just the sum of all these primitives — an explicit, editable, inspectable representation, in contrast to NeRF's opaque MLP weights.

Rendering is where the name comes from. Rather than marching a ray and querying a network at many sample points (NeRF's expensive approach), each 3D Gaussian is projected — splatted — onto the image plane, where it becomes a 2D Gaussian footprint. All the splats overlapping a pixel are sorted by depth and alpha-blended front to back, exactly the differentiable volume-compositing rule from volume rendering, but evaluated by fast GPU rasterization instead of ray-marching. This rasterized splatting is what delivers real-time frame rates (often well over 100 FPS) at high resolution.

Training mirrors NeRF's idea but on the explicit primitives. Starting from the sparse point cloud that structure-from-motion already produces, the system renders the current Gaussians, compares to the real photos, and backpropagates the image error to update every Gaussian's parameters. A clever adaptive-density control step periodically clones and splits Gaussians where detail is missing and prunes ones that are too transparent or redundant, so the count grows where the scene needs it. The trade-offs versus NeRF: faster training and real-time rendering and easy editing, at the cost of larger storage (millions of primitives) and, in the base method, some artifacts under extreme viewpoint or lighting changes — actively being addressed by anti-aliased, regularized, and dynamic-scene variants.