research
∙
06/02/2023
MLP-Mixer as a Wide and Sparse MLP
Multi-layer perceptron (MLP) is a fundamental component of deep learning...
share
research
∙
10/06/2022
Understanding Gradient Regularization in Deep Learning: Efficient Finite-Difference Computation and Implicit Bias
Gradient regularization (GR) is a method that penalizes the gradient nor...
research
∙
03/24/2021
Asymptotic Freeness of Layerwise Jacobians Caused by Invariance of Multilayer Perceptron: The Haar Orthogonal Case
Free Probability Theory (FPT) provides rich knowledge for handling mathe...
research
∙
02/26/2021
Layer-Wise Interpretation of Deep Neural Networks Using Identity Initialization
The interpretability of neural networks (NNs) is a challenging but essen...
research
∙
12/22/2020
Selective Forgetting of Deep Networks at a Finer Level than Samples
Selective forgetting or removing information from deep neural networks (...
research
∙
06/14/2020
The Spectrum of Fisher Information of Deep Networks Achieving Dynamical Isometry
The Fisher information matrix (FIM) is fundamental for understanding the...
research
∙
08/11/2019
Almost Surely Asymptotic Freeness for Jacobian Spectrum of Deep Network
Free probability theory helps us to understand Jacobian spectrum of deep...
research
∙
12/27/2018
Identifiability of parametric random matrix models
We investigate parameter identifiability of spectral distributions of ra...
research
∙
04/09/2018
Cauchy noise loss for stochastic optimization of random matrix models via free deterministic equivalents
Based on free probability theory and stochastic optimization, we introdu...
research
∙
10/25/2017