SESKStuff Everyone Should Know
SESK: Stuff Everyone Should Know · sesk.info/ai-rnn-carries-state

What lets a recurrent neural network carry information through a sequence?

Its next state uses the previous hidden state.

Start playing free

  1. A fresh random seed
  2. The previous hidden state ✔
  3. A fixed image filter

Why: At each time step, an RNN combines the current input with its earlier state. That recurrence gives later outputs access to information from earlier inputs.

Huh, didn't know that: The same transition function with the same parameters is reused at every time step.

The dad joke
The RNN remembered the punchline from the previous step.
Like this card?
Play the game

References

Answer: The previous hidden state. At each time step, an RNN combines the current input with its earlier state. That recurrence gives later outputs access to information from earlier inputs.

  1. Goodfellow, Bengio, Courville: Deep Learning, ch. 10 Sequence Modeling www.deeplearningbook.org/contents/rnn.html
    h(t) = f(h(t-1), x(t); θ)
    Checked 2026-10-10.

Think this answer is wrong? Tap "Challenge this answer" on the card in the game and tell us why.

Test yourself in the game

Artificial IntelligenceHow does backpropagation through time treat a recurrent network?Artificial IntelligenceWhy are ordinary RNN time steps difficult to parallelize within one sequence?Artificial IntelligenceWhat makes an attention mechanism self-attention?

That's stuff everyone should know.