Tag: Seq2seq
-
RNN to Transformer NMT: PyTorch Migration with 2.8x BLEU Gain
Stuck at BLEU 18 with GRU seq2seq? Here's the PyTorch code that hit BLEU 51 after migrating to Transformerโplus the causal mask bug that wasted 3 days.
-
LSTM Attention vs Self-Attention: How Bahdanau Evolved
Bahdanau's 2014 attention fixed seq2seq bottlenecks but kept sequential encoders. How three key problems led to self-attention and 3x speedups.