research
∙
07/31/2023
Predicting masked tokens in stochastic locations improves masked image modeling
Self-supervised learning is a promising paradigm in deep learning that e...
share
research
∙
04/14/2023
DINOv2: Learning Robust Visual Features without Supervision
The recent breakthroughs in natural language processing for model pretra...
research
∙
04/11/2023
A surprisingly simple technique to control the pretraining bias for better transfer: Expand or Narrow your representation
Self-Supervised Learning (SSL) models rely on a pretext task to learn re...
research
∙
01/23/2023
A Simple Recipe for Competitive Low-compute Self supervised Vision Models
Self-supervised methods in vision have been mostly focused on large arch...
research
∙
01/19/2023
Self-Supervised Learning from Images with a Joint-Embedding Predictive Architecture
This paper demonstrates an approach for learning highly semantic image r...
research
∙
12/10/2022
Uniform Masking Prevails in Vision-Language Pretraining
Masked Language Modeling (MLM) has proven to be an essential component o...
research
∙
11/03/2022
ImageNet-X: Understanding Model Mistakes with Factor of Variation Annotations
Deep learning vision systems are widely deployed across applications whe...
research
∙
10/13/2022
The Hidden Uniform Cluster Prior in Self-Supervised Learning
A successful paradigm in representation learning is to perform self-supe...
research
∙
06/01/2022
Cascaded Video Generation for Videos In-the-Wild
Videos can be created by first outlining a global view of the scene and ...
research
∙
04/14/2022
Masked Siamese Networks for Label-Efficient Learning
We propose Masked Siamese Networks (MSN), a self-supervised learning fra...
research
∙
12/31/2021
BARACK: Partially Supervised Group Robustness With Guarantees
While neural networks have shown remarkable success on classification ta...
research
∙
10/15/2021
Trade-offs of Local SGD at Scale: An Empirical Study
As datasets and models become increasingly large, distributed training h...
research
∙
06/04/2021
Hierarchical Video Generation for Complex Data
Videos can often be created by first outlining a global description of t...
research
∙
04/28/2021
Semi-Supervised Learning of Visual Features by Non-Parametrically Predicting View Assignments with Support Samples
This paper proposes a novel method of learning by predicting view assign...
research
∙
10/06/2020
A Closer Look at Codistillation for Distributed Training
Codistillation has been proposed as a mechanism to share knowledge among...
research
∙
06/22/2020
Revisiting Loss Modelling for Unstructured Pruning
By removing parameters from deep neural networks, unstructured pruning m...
research
∙
06/18/2020
Recovering Petaflops in Contrastive Semi-Supervised Learning of Visual Representations
We investigate a strategy for improving the computational efficiency of ...
research
∙
10/01/2019
SlowMo: Improving Communication-Efficient Distributed SGD with Slow Momentum
Distributed optimization is essential for training large models on large...
research
∙
08/16/2019
Needles in Haystacks: On Classifying Tiny Objects in Large Images
In some computer vision domains, such as medical or hyperspectral imagin...
research
∙
06/09/2019
Gossip-based Actor-Learner Architectures for Deep Reinforcement Learning
Multi-simulator training has contributed to the recent success of Deep R...
research
∙
04/27/2019
Improved Conditional VRNNs for Video Prediction
Predicting future frames for a video sequence is a challenging generativ...
research
∙
11/27/2018
Stochastic Gradient Push for Distributed Deep Learning
Large mini-batch parallel SGD is commonly used for distributed training ...
research
∙
07/13/2018
DNN's Sharpest Directions Along the SGD Trajectory
Recent work has identified that using a high learning rate or a small ba...
research
∙
06/20/2018
A Dissection of Overfitting and Generalization in Continuous Reinforcement Learning
The risks and perils of overfitting in machine learning are well known. ...
research
∙
06/11/2018
Fast Approximate Natural Gradient Descent in a Kronecker-factored Eigenbasis
Optimization algorithms that leverage gradient covariance information, s...
research
∙
11/13/2017
Three Factors Influencing Minima in SGD
We study the properties of the endpoint of stochastic gradient descent (...
research
∙
10/13/2017
Residual Connections Encourage Iterative Inference
Residual networks (Resnets) have become a prominent architecture in deep...
research
∙
06/16/2017
A Closer Look at Memorization in Deep Networks
We examine the role of memorization in deep learning, drawing connection...
research
∙
11/23/2016
A dataset and exploration of models for understanding video data through fill-in-the-blank question-answering
While deep convolutional neural networks frequently approach or exceed h...
research
∙
06/03/2016
Zoneout: Regularizing RNNs by Randomly Preserving Hidden Activations
We propose zoneout, a novel method for regularizing RNNs. At each timest...
research
∙
05/09/2016
Theano: A Python framework for fast computation of mathematical expressions
Theano is a Python library that allows to define, optimize, and evaluate...
research
∙
11/24/2015
Dynamic Capacity Networks
We introduce the Dynamic Capacity Network (DCN), a neural network that c...
research
∙
11/19/2015
Delving Deeper into Convolutional Networks for Learning Video Representations
We propose an approach to learn spatio-temporal features in videos from ...
research
∙
11/14/2015
Oracle performance for visual captioning
The task of associating images and videos with a natural language descri...
research
∙
02/27/2015
Describing Videos by Exploiting Temporal Structure
Recent progress in using recurrent neural networks (RNNs) for image desc...
research
∙
12/19/2014
