research
∙
07/11/2022
Exploring Length Generalization in Large Language Models
The ability to extrapolate from short problem instances to longer ones i...
share
research
∙
06/29/2022
Solving Quantitative Reasoning Problems with Language Models
Language models have achieved remarkable performance on a wide range of ...
research
∙
04/05/2022
PaLM: Scaling Language Modeling with Pathways
Large language models have been shown to achieve remarkable performance ...
research
∙
11/30/2021
Show Your Work: Scratchpads for Intermediate Computation with Language Models
Large pre-trained language models perform remarkably well on tasks that ...
research
∙
10/27/2020
Are wider nets better given the same number of parameters?
Empirical studies demonstrate that the performance of neural networks im...
research
∙
06/15/2020
On the training dynamics of deep networks with L_2 regularization
We study the role of L_2 regularization in deep learning, and uncover si...
research
∙
06/11/2020
On the asymptotics of wide networks with polynomial activations
We consider an existing conjecture addressing the asymptotic behavior of...
research
∙
03/04/2020
The large learning rate phase of deep learning: the catapult mechanism
The choice of initial learning rate can have a profound effect on the pe...
research
∙
09/25/2019
Wider Networks Learn Better Features
Transferability of learned features between tasks can massively reduce t...
research
∙
09/25/2019
Asymptotics of Wide Networks from Feynman Diagrams
Understanding the asymptotic behavior of wide networks is of considerabl...
research
∙
12/12/2018