Discourse-Based Evaluation of Language Understanding

07/19/2019
by   Damien Sileo, et al.
0

We introduce DiscEval, a compilation of 11 evaluation datasets with a focus on discourse, that can be used for evaluation of English Natural Language Understanding when considering meaning as use. We make the case that evaluation with discourse tasks is overlooked and that Natural Language Inference (NLI) pretraining may not lead to the learning really universal representations. DiscEval can also be used as supplementary training data for multi-task learning-based systems, and is publicly available, alongside the code for gathering and preprocessing the datasets.

READ FULL TEXT

Please sign up or login with your details

Forgot password? Click here to reset