Preventing RNN from Using Sequence Length as a Feature

12/16/2022
by   Jean-Thomas Baillargeon, et al.
0

Recurrent neural networks are deep learning topologies that can be trained to classify long documents. However, in our recent work, we found a critical problem with these cells: they can use the length differences between texts of different classes as a prominent classification feature. This has the effect of producing models that are brittle and fragile to concept drift, can provide misleading performances and are trivially explainable regardless of text content. This paper illustrates the problem using synthetic and real-world data and provides a simple solution using weight decay regularization.

READ FULL TEXT

Please sign up or login with your details

Forgot password? Click here to reset