recurrent neural network
/ rih-KUR-unt NUR-ul NET-wurk /
A recurrent neural network, or RNN, is a network built to read things in order — word after word, note after note, day after day. Most networks treat their input as one frozen snapshot. But a sentence or a melody only makes sense as a sequence, where what came before colors what comes next. An RNN handles this by reading one step at a time and keeping a running memory, called the hidden state — a little scratchpad it updates at every step and carries forward, so the meaning of earlier words can shape its reading of later ones.
Picture reading a sentence aloud with a single sticky note in hand. You read the first word, jot a quick summary; read the next word, update the note to fold in what you just learned; and so on to the end. The note never grows — it just gets rewritten each step — yet by the last word it carries a compressed trace of the whole sentence. That looping update, the same rule applied at every step, is what "recurrent" means. It lets one modest network handle sequences of any length, from three words to three thousand.
RNNs powered the first wave of good machine translation and speech recognition, but they have a stubborn weakness. Because the memory is squeezed and rewritten at every step, the influence of early words tends to fade — the vanishing gradient problem — so a plain RNN struggles to connect things that are far apart, like a pronoun and the noun it refers to ten sentences back. That limit drove the invention of gated variants like LSTM and GRU, and eventually the Transformer, which reads a whole sequence at once and has largely displaced RNNs for large-scale language work.
Read "I grew up in France, so I speak fluent ___." To fill the blank with "French," the network must still remember "France" from the start of the sentence — exactly the long-range link plain RNNs find hardest.
The whole point — and the weakness — of an RNN is carrying meaning across the gap.
An RNN is really one small network applied over and over, sharing the same weights at every step — not a long chain of different layers. Its old crown for sequence tasks has largely passed to the Transformer, but the core idea of a carried-forward memory remains foundational.