research
∙
05/23/2023
Perception Test: A Diagnostic Benchmark for Multimodal Video Models
We propose a novel multimodal video benchmark - the Perception Test - to...
share
research
∙
05/06/2022
CLIP-CLOP: CLIP-Guided Collage and Photomontage
The unabated mystique of large-scale neural networks, such as the CLIP d...
research
∙
03/29/2022
Training Compute-Optimal Large Language Models
We investigate the optimal model size and number of tokens for training ...
research
∙
02/17/2022
Retrieval-Augmented Reinforcement Learning
Most deep reinforcement learning (RL) algorithms distill experience into...
research
∙
02/02/2022
Unified Scaling Laws for Routed Language Models
The performance of a language model has been shown to be effectively mod...
research
∙
12/08/2021
Improving language models by retrieving from trillions of tokens
We enhance auto-regressive language models by conditioning on document c...
research
∙
06/07/2021
Top-KAST: Top-K Always Sparse Training
Sparse neural networks are becoming increasingly important as the field ...
research
∙
05/01/2021
Generative Art Using Neural Visual Grammars and Dual Encoders
Whilst there are perhaps only a few scientific methods, there seem to be...
research
∙
11/05/2020
Contrastive Topographic Models: Energy-based density models applied to the understanding of sensory coding and cortical topography
We address the problem of building theoretical models that help elucidat...
research
∙
10/06/2020
From Language Games to Drawing Games
We attempt to automate various artistic processes by inventing a set of ...
research
∙
09/26/2020
Small Data, Big Decisions: Model Selection in the Small-Data Regime
Highly overparametrized neural networks can display curiously strong gen...
research
∙
06/12/2020
AlgebraNets
Neural networks have historically been built layerwise from the set of f...
research
∙
06/12/2020
A Practical Sparse Approximation for Real Time Recurrent Learning
Current methods for training recurrent neural networks are based on back...
research
∙
12/16/2019
A Deep Neural Network's Loss Surface Contains Every Low-dimensional Pattern
The work "Loss Landscape Sightseeing with Multi-Point Optimization" (Sko...
research
∙
12/14/2019
Adapting Behaviour for Learning Progress
Determining what experience to generate to best facilitate learning (i.e...
research
∙
10/07/2019
Meta-Learning Deep Energy-Based Memory Models
We study the problem of learning associative memory – a system which is ...
research
∙
05/08/2019
Meta-learning of Sequential Strategies
In this report we review memory-based meta-learning as a tool for buildi...
research
∙
02/06/2019
Distilling Policy Distillation
The transfer of knowledge from one policy to another is an important too...
research
∙
07/16/2018
Meta-Learning with Latent Embedding Optimization
Gradient-based meta-learning techniques are both widely applicable and p...
research
∙
06/11/2018
Massively Parallel Video Networks
We introduce a class of causal video understanding models that aims to i...
research
∙
06/06/2018
Meta Learning by the Baldwin Effect
The scope of the Baldwin effect was recently called into question by two...
research
∙
06/05/2018
Mix&Match - Agent Curricula for Reinforcement Learning
We introduce Mix&Match (M&M) - a training framework designed to facilita...
research
∙
03/10/2018
Kickstarting Deep Reinforcement Learning
We present a method for using previously-trained 'teacher' agents to kic...
research
∙
11/27/2017
Population Based Training of Neural Networks
Neural networks dominate the modern machine learning landscape, but thei...
research
∙
11/01/2017
Beautiful and damned. Combined effect of content quality and social ties on user engagement
User participation in online communities is driven by the intertwinement...
research
∙
03/01/2017
Understanding Synthetic Gradients and Decoupled Neural Interfaces
When training neural networks, the use of Synthetic Gradients (SG) allow...
research
∙
08/18/2016
Decoupled Neural Interfaces using Synthetic Gradients
Training directed neural networks typically requires forward-propagating...
research
∙
06/15/2016
Strategic Attentive Writer for Learning Macro-Actions
We present a novel deep recurrent neural network architecture that learn...
research
∙
03/09/2016
Recursive Recurrent Nets with Attention Modeling for OCR in the Wild
We present recursive recurrent neural networks with attention modeling (...
research
∙
12/13/2015
Cross-dimensional Weighting for Aggregated Deep Convolutional Features
We propose a simple and straightforward way of creating powerful image r...
research
∙
11/06/2014