research
∙
09/21/2023
Boolformer: Symbolic Regression of Logic Functions with Transformers
In this work, we introduce Boolformer, the first Transformer architectur...
share
research
∙
06/27/2023
Length Generalization in Arithmetic Transformers
We examine how transformers cope with two challenges: learning basic int...
research
∙
04/22/2022
End-to-end symbolic regression with transformers
Symbolic regression, the task of predicting the mathematical expression ...
research
∙
02/09/2022
Optimal learning rate schedules in high-dimensional non-convex optimization problems
Learning rate schedules are ubiquitously used to speed up and improve op...
research
∙
01/12/2022
Deep Symbolic Regression for Recurrent Sequences
Symbolic regression, i.e. predicting a function from the observation of ...
research
∙
06/10/2021
Transformed CNNs: recasting pre-trained convolutional layers with self-attention
Vision Transformers (ViT) have recently emerged as a powerful alternativ...
research
∙
03/19/2021
ConViT: Improving Vision Transformers with Soft Convolutional Inductive Biases
Convolutional architectures have proven extremely successful for vision ...
research
∙
03/09/2021
More data or more parameters? Investigating the effect of data structure on generalization
One of the central features of deep learning is the generalization abili...
research
∙
11/24/2020
The dynamics of learning with feedback alignment
Direct Feedback Alignment (DFA) is emerging as an efficient and biologic...
research
∙
11/03/2020
Conditioned Text Generation with Transfer for Closed-Domain Dialogue Systems
Scarcity of training data for task-oriented dialogue systems is a well k...
research
∙
06/05/2020
Triple descent and the two kinds of overfitting: Where why do they appear?
A recent line of research has highlighted the existence of a double desc...
research
∙
03/02/2020
Double Trouble in Double Descent : Bias and Variance(s) in the Lazy Regime
Deep neural networks can achieve remarkable generalization performances ...
research
∙
11/09/2019
Conditioned Query Generation for Task-Oriented Dialogue Systems
Scarcity of training data for task-oriented dialogue systems is a well k...
research
∙
06/16/2019
Finding the Needle in the Haystack with Convolutions: on the benefits of architectural bias
Despite the phenomenal success of deep neural networks in a broad range ...
research
∙
01/06/2019
Scaling description of generalization with number of parameters in deep learning
We provide a description for the evolution of the generalization perform...
research
∙
10/22/2018
A jamming transition from under- to over-parametrization affects loss landscape and generalization
We argue that in fully-connected networks a phase transition delimits th...
research
∙
09/25/2018