Knowledge distillation is often used to transfer knowledge from a strong...
Sampling proper negatives from a large document pool is vital to effecti...
Knowledge distillation is an effective way to transfer knowledge from a
...
We demonstrate that explicitly aligning the pretraining objectives to th...