Memory-enhanced Decoder for Neural Machine Translation
share
We propose to enhance the RNN decoder in a neural machine translator (NMT) with external memory, as a natural but powerful extension to the state in the decoding RNN. This memory-enhanced RNN decoder is called MemDec. At each time during decoding, MemDec will read from this memory and write to this memory once, both with content-based addressing. Unlike the unbounded memory in previous workRNNsearch to store the representation of source sentence, the memory in MemDec is a matrix with pre-determined size designed to better capture the information important for the decoding process at each time step. Our empirical study on Chinese-English translation shows that it can improve by 4.8 BLEU upon Groundhog and 5.3 BLEU upon on Moses, yielding the best performance achieved with the same training set.
READ FULL TEXT