research
∙
07/03/2023
Analyzing and Improving Greedy 2-Coordinate Updates for Equality-Constrained Optimization via Steepest Descent in the 1-Norm
We consider minimizing a smooth function subject to a summation constrai...
share
research
∙
06/22/2023
Don't be so Monotone: Relaxing Stochastic Line Search in Over-Parameterized Models
Recent works have shown that line search methods can speed up Stochastic...
research
∙
06/05/2023
Searching for Optimal Per-Coordinate Step-sizes with Multidimensional Backtracking
The backtracking line-search is an effective technique to automatically ...
research
∙
05/30/2023
BiSLS/SPS: Auto-tune Step Sizes for Stable Bi-level Optimization
The popularity of bi-level optimization (BO) in deep learning has spurre...
research
∙
04/27/2023
Noise Is Not the Main Factor Behind the Gap Between SGD and Adam on Transformers, but Sign Descent Might Be
The success of the Adam optimizer on a wide array of architectures has m...
research
∙
04/02/2023
Fast Convergence of Random Reshuffling under Over-Parameterization and the Polyak-Łojasiewicz Condition
Modern machine learning models are often over-parameterized and as a res...
research
∙
02/20/2023
Simplifying Momentum-based Riemannian Submanifold Optimization
Riemannian submanifold optimization with momentum is computationally cha...
research
∙
02/06/2023
Target-based Surrogates for Stochastic Optimization
We consider minimizing functions for which it is expensive to compute th...
research
∙
07/29/2022
Improved Policy Optimization for Online Imitation Learning
We consider online imitation learning (OIL), where the task is to find a...
research
∙
07/22/2021
Structured second-order methods via natural gradient descent
In this paper, we propose new structured second-order methods and struct...
research
∙
02/18/2021
SVRG Meets AdaGrad: Painless Variance Reduction
Variance reduction (VR) methods for finite-sum minimization typically re...
research
∙
02/15/2021
Tractable structured natural gradient descent using local parameterizations
Natural-gradient descent on structured parameter spaces (e.g., low-rank ...
research
∙
01/11/2021
The pathway elaboration method for mean first passage time estimation in large continuous-time Markov chains with applications to nucleic acid kinetics
Continuous-time Markov chains (CTMCs) are widely used in many applicatio...
research
∙
12/31/2020
Robust Asymmetric Learning in POMDPs
Policies for partially observed Markov decision processes can be efficie...
research
∙
11/02/2020
Homeomorphic-Invariance of EM: Non-Asymptotic Convergence in KL Divergence for Exponential Families via Mirror Descent
Expectation maximization (EM) is the default algorithm for fitting proba...
research
∙
10/22/2020
Regret Bounds without Lipschitz Continuity: Online Learning with Relative-Lipschitz Losses
In online convex optimization (OCO), Lipschitz continuity of the functio...
research
∙
10/02/2020
Variance-Reduced Methods for Machine Learning
Stochastic optimization lies at the heart of machine learning, and its c...
research
∙
06/11/2020
Adaptive Gradient Methods Converge Faster with Over-Parameterization (and you can do a line-search)
As adaptive gradient methods are typically used for training over-parame...
research
∙
02/24/2020
Handling the Positive-Definite Constraint in the Bayesian Learning Rule
Bayesian learning rule is a recently proposed variational inference meth...
research
∙
10/29/2019
Stein's Lemma for the Reparameterization Trick with Exponential Family Mixtures
Stein's method (Stein, 1973; 1981) is a powerful tool for statistical ap...
research
∙
10/11/2019
Fast and Furious Convergence: Stochastic Second Order Methods under Interpolation
We consider stochastic second order methods for minimizing strongly-conv...
research
∙
07/08/2019
P4-IPsec: Implementation of IPsec Gateways in P4 with SDN Control for Host-to-Site Scenarios
In this paper we propose P4-IPsec which follows the software-defined net...
research
∙
07/08/2019
xRAC: Execution and Access Control for Restricted Application Containers on Managed Hosts
We propose xRAC to permit users to run special applications on managed h...
research
∙
07/02/2019
Where are the Masks: Instance Segmentation with Image-level Supervision
A major obstacle in instance segmentation is that existing methods often...
research
∙
06/14/2019
Instance Segmentation with Point Supervision
Instance segmentation methods often require costly per-pixel labels. We ...
research
∙
06/07/2019
Fast and Simple Natural-Gradient Variational Inference with Mixture of Exponential-family Approximations
Natural-gradient methods enable fast and simple algorithms for variation...
research
∙
05/24/2019
Painless Stochastic Gradient: Interpolation, Line-Search, and Convergence Rates
Recent works have shown that stochastic gradient descent (SGD) achieves ...
research
∙
05/16/2019
Efficient Deep Gaussian Process Models for Variable-Sized Input
Deep Gaussian processes (DGP) have appealing Bayesian properties, can ha...
research
∙
04/15/2019
P4-MACsec: Dynamic Topology Monitoring and Data Layer Protection with MACsec in P4-SDN
We propose P4-MACsec to protect network links between P4 switches throug...
research
∙
03/20/2019
Distributed Maximization of "Submodular plus Diversity" Functions for Multi-label Feature Selection on Huge Datasets
There are many problems in machine learning and data mining which are eq...
research
∙
11/11/2018
SLANG: Fast Structured Covariance Approximations for Bayesian Deep Learning with Natural Gradient
Uncertainty estimation in large deep-learning models is a computationall...
research
∙
10/16/2018
Fast and Faster Convergence of SGD for Over-Parameterized Models and an Accelerated Perceptron
Modern machine learning focuses on highly expressive models that are abl...
research
∙
10/10/2018
Combining Bayesian Optimization and Lipschitz Optimization
Bayesian optimization and Lipschitz optimization have developed alternat...
research
∙
09/13/2018
Does Your Model Know the Digit 6 Is Not a Cat? A Less Biased Evaluation of "Outlier" Detectors
In the real world, a learning system could receive an input that looks n...
research
∙
07/25/2018
Where are the Blobs: Counting by Localization with Point Supervision
Object counting is an important task in computer vision due to its growi...
research
∙
05/24/2018
New Insights into Bootstrapping for Bandits
We investigate the use of bootstrapping in the bandit setting. We first ...
research
∙
03/14/2017
Online Learning Rate Adaptation with Hypergradient Descent
We introduce a general method for improving the convergence rate of grad...
research
∙
12/13/2016
Fast Patch-based Style Transfer of Arbitrary Style
Artistic style transfer is an image synthesis problem where the content ...
research
∙
08/16/2016
Linear Convergence of Gradient and Proximal-Gradient Methods Under the Polyak-Łojasiewicz Condition
In 1963, Polyak proposed a simple condition that is sufficient to show a...
research
∙
08/05/2016
Play and Learn: Using Video Games to Train Computer Vision Models
Video games are a compelling source of annotated data as they can readil...
research
∙
11/05/2015
Stop Wasting My Gradients: Practical SVRG
We present and analyze several strategies for improving the performance ...
research
∙
10/31/2015
Faster Stochastic Variational Inference using Proximal-Gradient Methods with General Divergence Functions
Several recent works have explored stochastic gradient methods for varia...
research
∙
06/01/2015
Coordinate Descent Converges Faster with the Gauss-Southwell Rule Than Random Selection
There has been significant recent work on the theory and application of ...
research
∙
04/16/2015
Non-Uniform Stochastic Average Gradient Method for Training Conditional Random Fields
We apply stochastic average gradient (SAG) algorithms for training condi...
research
∙
02/27/2015
Influence Maximization with Bandits
We consider the problem of influence maximization, the problem of maximi...
research
∙
02/06/2015
Hierarchical Maximum-Margin Clustering
We present a hierarchical maximum-margin clustering method for unsupervi...
research
∙
11/04/2014
Convex Optimization for Big Data
This article reviews recent advances in convex optimization algorithms f...
research
∙
09/10/2013
Minimizing Finite Sums with the Stochastic Average Gradient
We propose the stochastic average gradient (SAG) method for optimizing t...
research
∙
12/10/2012
A simpler approach to obtaining an O(1/t) convergence rate for the projected stochastic subgradient method
In this note, we present a new averaging technique for the projected sto...
research
∙
07/19/2012