research
∙
05/17/2022
Universal characteristics of deep neural network loss surfaces from random matrix theory
This paper considers several aspects of random matrix universality in de...
share
research
∙
02/12/2021
Applicability of Random Matrix Theory in Deep Learning
We investigate the local spectral statistics of the loss surface Hessian...
research
∙
11/15/2020
Explaining the Adaptive Generalisation Gap
We conjecture that the reason for the difference in generalisation betwe...
research
∙
06/16/2020
Curvature is Key: Sub-Sampled Loss Surfaces and the Implications for Large Batch Training
We study the effect of mini-batching on the loss landscape of deep neura...
research
∙
06/16/2020
Flatness is a False Friend
Hessian based measures of flatness, such as the trace, Frobenius and spe...
research
∙
06/13/2020
Beyond Random Matrix Theory for Deep Networks
We investigate whether the Wigner semi-circle and Marcenko-Pastur distri...
research
∙
03/02/2020
Iterate Averaging Helps: An Alternative Perspective in Deep Learning
Iterate averaging has a rich history in optimisation, but has only very ...
research
∙
12/20/2019
MLRG Deep Curvature
We present MLRG Deep Curvature suite, a PyTorch-based, open-source packa...
research
∙
12/19/2019
A Maximum Entropy approach to Massive Graph Spectra
Graph spectral techniques for measuring graph similarity, or for learnin...
research
∙
06/03/2019
MEMe: An Accurate Maximum Entropy Method for Efficient Approximations in Large-Scale Machine Learning
Efficient approximation lies at the heart of large-scale machine learnin...
research
∙
04/18/2018
Entropic Spectral Learning in Large Scale Networks
We present a novel algorithm for learning the spectral density of large ...
research
∙
02/21/2018
VBALD - Variational Bayesian Approximation of Log Determinants
Evaluating the log determinant of a positive definite matrix is ubiquito...
research
∙
09/08/2017
Entropic Determinants
The ability of many powerful machine learning algorithms to deal with la...
research
∙
04/24/2017