Feed-Forward Networks with Attention Can Solve Some Long-Term Memory Problems

12/29/2015
by   Colin Raffel, et al.
0

We propose a simplified model of attention which is applicable to feed-forward neural networks and demonstrate that the resulting model can solve the synthetic "addition" and "multiplication" long-term memory problems for sequence lengths which are both longer and more widely varying than the best published results for these tasks.

READ FULL TEXT

Please sign up or login with your details

Forgot password? Click here to reset