A ten-minute introduction to sequence-to-sequence learning in Keras

Abstraction: Keras LSTM encoder-decoder implementation for sequence-to-sequence translation

Key points:

• Canonical Seq2Seq: encoder RNN reads full input, discards its output, passes final hidden state as initial state to decoder RNN
• Teacher forcing: decoder trained on ground-truth shifted target sequence (targets[t+1] given targets[...t]) rather than its own previous predictions
• Inference loop: encode input → start token → argmax sample → append → repeat until end-of-sequence token or length limit
• Key Keras arguments: return_state=True exposes encoder's final states; initial_state injects them into decoder; return_sequences=True outputs full sequence for stacked layers
• Character-level English→French demo on manythings.org/anki dataset; word-level variant uses Embedding layer on integer sequences
• GRU adaptation is simpler — one state instead of LSTM's two (h and c)

Connections: Keras · Tensorflow · Sequence To Sequence · Recurrent Neural Networks · Encoder Decoder

Source: https://blog.keras.io/a-ten-minute-introduction-to-sequence-to-sequence-learning-in-keras.html