trajectory transformer (TT)

Where Decision Transformer predicts only the next action, Trajectory Transformer models everything — states, actions, and rewards alike — as one long sequence, becoming a full model of how trajectories unfold. It is offline RL recast as sequence modeling plus planning.

It discretizes and tokenizes entire trajectories and trains a transformer to predict the next token, learning the joint distribution over future states, actions, and rewards. At decision time it plans by beam search over this learned model, searching for high-reward continuations rather than reading off a single conditioned action.

By planning inside a learned trajectory model, it can do the credit assignment and stitching that pure return-conditioning misses — at the cost of expensive sequence-level search. It sits right on the border between offline RL and model-based RL.