importance-weighted autoencoder (IWAE)

A variational autoencoder optimizes a single-sample lower bound that can be loose when the approximate posterior is a poor match for the true one. The importance-weighted autoencoder tightens that bound simply by averaging over several latent samples before taking the logarithm. Intuitively, drawing many guesses and weighting them by how well each explains the data lets a few lucky samples rescue the estimate, so the model is judged less harshly for an imperfect posterior.

Formally the objective is the log of the average importance weight over multiple latent draws, where each weight is the ratio of the joint density to the proposal. This multi-sample bound is provably at least as tight as the standard ELBO and grows monotonically tighter as the number of samples increases, converging to the true log-likelihood in the limit. The improvement comes at a cost: more samples per step, and the per-sample gradient signal to the encoder actually weakens as the count grows, which can slow inference-network learning unless corrected.

IWAE is the canonical demonstration that the ELBO is just one member of a family of valid likelihood bounds, opening the door to tighter Monte-Carlo objectives that trade extra computation for a more faithful likelihood estimate.