joint-embedding predictive architecture (JEPA)

JEPA is a design philosophy, championed as a route to more world-model-like learning, that says: do not predict the raw signal, predict its representation. A generative model that reconstructs every pixel wastes capacity modeling unpredictable detail like exact textures and noise. JEPA instead asks a network to predict the abstract embedding of a missing part of the input from the embedding of the visible part, so it can ignore details that carry no useful structure.

In the image instantiation, I-JEPA, a context encoder embeds a set of visible patches, a target encoder (a momentum copy) embeds several masked target blocks, and a predictor takes the context embeddings plus positional information about a target block and predicts that block's target embedding, with the loss measured in feature space, not pixel space. As in BYOL and DINO, the target encoder is updated by exponential moving average with a stop-gradient, which is what stops the system from collapsing to a constant. Sampling fairly large, semantically meaningful target blocks (rather than scattered pixels) is important for learning good abstractions.

The motivation is that predicting in representation space is naturally invariant to nuisance variability and avoids the pixel-reconstruction tax that MAE-style models pay, while keeping the strong feature quality of self-distillation. JEPA generalizes beyond images to video and other modalities, and is positioned as a building block for predictive world models.