research
∙
07/18/2023
Optimistic Estimate Uncovers the Potential of Nonlinear Models
We propose an optimistic estimate to evaluate the best possible fitting ...
share
research
∙
05/25/2023
Stochastic Modified Equations and Dynamics of Dropout Algorithm
Dropout is a widely utilized regularization technique in the training of...
research
∙
05/20/2023
Loss Spike in Training Neural Networks
In this work, we study the mechanism underlying loss spikes observed dur...
research
∙
11/21/2022
Linear Stability Hypothesis and Rank Stratification for Nonlinear Models
Models with nonlinear architectures/parameterizations such as deep neura...
research
∙
07/13/2022
Implicit regularization of dropout
It is important to understand how the popular regularization method drop...
research
∙
11/30/2021
Embedding Principle: a hierarchical structure of loss landscape of deep neural networks
We prove a general Embedding Principle of loss landscape of deep neural ...
research
∙
11/01/2021
A variance principle explains why dropout finds flatter minima
Although dropout has achieved great success in deep learning, little is ...
research
∙
05/30/2021