Vocabulary Transfer for Medical Texts

08/04/2022
by   Vladislav D. Mosin, et al.
0

Vocabulary transfer is a transfer learning subtask in which language models fine-tune with the corpus-specific tokenization instead of the default one, which is being used during pretraining. This usually improves the resulting performance of the model, and in the paper, we demonstrate that vocabulary transfer is especially beneficial for medical text processing. Using three different medical natural language processing datasets, we show vocabulary transfer to provide up to ten extra percentage points for the downstream classifier accuracy.

READ FULL TEXT

Please sign up or login with your details

Forgot password? Click here to reset