frame interpolation

Frame interpolation synthesizes brand-new frames in between the real ones, so a video shot at 24 or 30 fps can be smoothly converted to 60 or 120 fps, or slowed into crisp slow-motion. The fundamental challenge is that you are inventing pixels that were never captured: to place content correctly at an intermediate time, you must figure out how everything moved between the two surrounding frames and then render the in-between view consistently.

Two main strategies exist. Flow-based methods estimate bidirectional optical flow between the two input frames, then warp both frames to the target time t (e.g., t = 0.5 for the midpoint) and blend them; advances here include linear and quadratic motion models, and softmax splatting (forward warping that resolves which pixel wins when several map to the same place). Kernel-based methods (AdaConv, SepConv) instead learn a spatially-varying convolution kernel per output pixel, generating the new pixel directly from a neighborhood without an explicit flow, which handles small motions and blur gracefully but struggles with large displacement. Recent systems combine flow and synthesis with transformers or diffusion priors. Widely used models include DAIN (depth-aware), the fast real-time RIFE, and FILM (designed for large motion and near-duplicate photos).

The failure modes are instructive. Occlusion and disocclusion create holes, regions visible in only one of the two frames, that pure warping cannot fill correctly. Large and nonlinear motion breaks the constant-velocity assumption behind simple linear blending, producing ghosting and tearing. Motion blur and brightness changes further violate the matching assumptions. Quality is reported with PSNR and SSIM for fidelity and LPIPS for perceptual realism, but the visible giveaways of a bad interpolation are ghost limbs, melted edges, and stutter around fast-moving objects.