SESKStuff Everyone Should Know
SESK: Stuff Everyone Should Know · sesk.info/ai-bptt-unrolls-time

How does backpropagation through time treat a recurrent network?

It treats the RNN as an unrolled deep network.

Start playing free

  1. As one static lookup table
  2. As unrelated classifiers
  3. As a network unrolled across time ✔

Why: Each time step becomes a copy in an unrolled computation graph. Ordinary backpropagation then passes gradients backward through those linked copies.

Huh, didn't know that: All unrolled copies represent repeated uses of the same recurrent parameters.

The dad joke
Backpropagation through time is the only kind of time travel that needs a chain rule.
Like this card?
Play the game

References

Answer: As a network unrolled across time. Each time step becomes a copy in an unrolled computation graph. Ordinary backpropagation then passes gradients backward through those linked copies.

  1. Goodfellow, Bengio, Courville: Deep Learning, ch. 10 Sequence Modeling www.deeplearningbook.org/contents/rnn.html
    The back-propagation algorithm applied to the unrolled graph with O(τ) cost is called back-propagation through time (BPTT)
    Checked 2026-10-10.

Think this answer is wrong? Tap "Challenge this answer" on the card in the game and tell us why.

Test yourself in the game

Artificial IntelligenceWhat lets a recurrent neural network carry information through a sequence?Artificial IntelligenceWhy are ordinary RNN time steps difficult to parallelize within one sequence?Artificial IntelligenceHow do AI, machine learning and deep learning fit together?

That's stuff everyone should know.