Diverse Lottery Tickets Boost Ensemble from a Single Pretrained Model

05/24/2022
by   Sosuke Kobayashi, et al.
20

Ensembling is a popular method used to improve performance as a last resort. However, ensembling multiple models finetuned from a single pretrained model has been not very effective; this could be due to the lack of diversity among ensemble members. This paper proposes Multi-Ticket Ensemble, which finetunes different subnetworks of a single pretrained model and ensembles them. We empirically demonstrated that winning-ticket subnetworks produced more diverse predictions than dense networks, and their ensemble outperformed the standard ensemble on some tasks.

READ FULL TEXT

Please sign up or login with your details

Forgot password? Click here to reset