Transformers to Learn Hierarchical Contexts in Multiparty Dialogue for Span-based Question Answering

04/07/2020
by   Changmao Li, et al.
0

We introduce a novel approach to transformers that learns hierarchical representations in multiparty dialogue. First, three language modeling tasks are used to pre-train the transformers, token- and utterance-level language modeling and utterance order prediction, that learn both token and utterance embeddings for better understanding in dialogue contexts. Then, multi-task learning between the utterance prediction and the token span prediction is applied to fine-tune for span-based question answering (QA). Our approach is evaluated on the FriendsQA dataset and shows improvements of 3.8 the two state-of-the-art transformer models, BERT and RoBERTa, respectively.

READ FULL TEXT

Please sign up or login with your details

Forgot password? Click here to reset