GRU

A GRU — gated recurrent unit — is a streamlined cousin of the LSTM. It chases the same goal: let a recurrent network carry information across long stretches of a sequence without it fading away. The difference is that the GRU does it with fewer moving parts. If the LSTM is a machine with three valves and two separate memory lines, the GRU is a tidier design with two valves and a single memory, aiming for nearly the same result with less to build and train.

It works through two gates. The update gate decides how much of the old memory to keep versus how much new information to let in — in effect blending "hold on to what I knew" against "take in what just arrived" in one move. The reset gate decides how much of the past to ignore when forming a fresh candidate memory, which helps the unit shed irrelevant history. Crucially, the GRU merges the LSTM's separate cell state and hidden state into one, so it has fewer numbers to learn. Like the LSTM, its gates can choose to leave memory largely untouched, which is what lets useful signals — and training's error signals — survive across many steps instead of vanishing.

In practice GRUs and LSTMs perform similarly on most tasks; neither is universally better, so people often try both. The GRU's appeal is that it is lighter and a bit faster to train, which matters on smaller datasets or tighter hardware. The honest caveat is the same one that applies to the whole RNN family: they read one step at a time, which makes them slow on very long sequences, and for the largest language models the Transformer has overtaken them. For modest sequence problems, though, a GRU remains a practical, economical workhorse.