Galactica: A Large Language Model for Science

11/16/2022
by   Ross Taylor, et al.
13

Information overload is a major obstacle to scientific progress. The explosive growth in scientific literature and data has made it ever harder to discover useful insights in a large mass of information. Today scientific knowledge is accessed through search engines, but they are unable to organize scientific knowledge alone. In this paper we introduce Galactica: a large language model that can store, combine and reason about scientific knowledge. We train on a large scientific corpus of papers, reference material, knowledge bases and many other sources. We outperform existing models on a range of scientific tasks. On technical knowledge probes such as LaTeX equations, Galactica outperforms the latest GPT-3 by 68.2 performs well on reasoning, outperforming Chinchilla on mathematical MMLU by 41.3 also sets a new state-of-the-art on downstream tasks such as PubMedQA and MedMCQA dev of 77.6 corpus, Galactica outperforms BLOOM and OPT-175B on BIG-bench. We believe these results demonstrate the potential for language models as a new interface for science. We open source the model for the benefit of the scientific community.

READ FULL TEXT

Please sign up or login with your details

Forgot password? Click here to reset