LSTM
/ el-es-tee-EM /
LSTM stands for long short-term memory, and it is a smarter kind of recurrent network built to fix the plain RNN's forgetfulness. A basic RNN crams its memory into one note that gets overwritten at every step, so anything from early on quickly washes out. An LSTM instead keeps a protected memory line — the cell state — that information can ride along almost untouched, like a conveyor belt running through the whole sequence, picking up and dropping off cargo only when told to.
What does the telling is a set of small valves called gates, each a tiny learned controller that outputs a number between 0 (fully shut) and 1 (fully open). The forget gate decides what old memory to wipe; the input gate decides what new information is worth storing; the output gate decides how much of the memory to reveal at the current step. Because these gates can choose to leave the cell state alone, a fact learned at word three can survive untouched to word three hundred — and crucially, the error signal during training flows back along that protected line without fading, sidestepping the vanishing-gradient trap that crippled plain RNNs.
From the late 1990s until the rise of the Transformer around 2017, LSTMs were the workhorse of sequence learning, behind a generation of machine translation, speech recognition, and text prediction. They are not magic: the gating machinery is heavier to train, they still read one step at a time (so they are slow on long inputs and hard to parallelize), and the Transformer has overtaken them for the largest language models. But for modest, ordered data — sensor streams, time series, on-device tasks — an LSTM is still a sound, efficient choice.
Reading "The keys to the cabinet are on the table," an LSTM can store "keys" (plural) on its protected cell state, hold it across the words in between, and so correctly pick "are" rather than "is" — a long-range agreement a plain RNN often fumbles.
Gates let one key fact ride the memory line, untouched, across many words.
The "long short-term" name is not a typo: it is a memory that holds short-term information for a long time. The cell state is the protected highway; the gates are the on- and off-ramps that decide what gets on, stays, or leaves.