research
∙
09/15/2023
Augmenting conformers with structured state space models for online speech recognition
Online speech recognition, where the model only accesses context to the ...
share
research
∙
03/11/2023
Resurrecting Recurrent Neural Networks for Long Sequences
Recurrent Neural Networks (RNNs) offer fast inference on long sequences ...
research
∙
03/07/2023
Structured State Space Models for In-Context Reinforcement Learning
Structured state space sequence (S4) models have recently achieved state...
research
∙
01/25/2023
Modelling Long Range Dependencies in N-D: From Task-Specific to a General Purpose CNN
Performant Convolutional Neural Network (CNN) architectures must be tail...
research
∙
12/20/2022
Pretraining Without Attention
Transformers have been essential to pretraining success in NLP. Other ar...
research
∙
10/12/2022
S4ND: Modeling Images and Videos as Multidimensional Signals Using State Spaces
Visual data such as images and videos are typically modeled as discretiz...
research
∙
06/24/2022
How to Train Your HiPPO: State Space Models with Generalized Orthogonal Basis Projections
Linear time-invariant state space models (SSM) are a classical model fro...
research
∙
06/23/2022
On the Parameterization and Initialization of Diagonal State Space Models
State space models (SSM) have recently been shown to be very effective a...
research
∙
06/07/2022
Towards a General Purpose CNN for Long Range Dependencies in ND
The use of Convolutional Neural Networks (CNNs) is widespread in Deep Le...
research
∙
02/20/2022
It's Raw! Audio Generation with State-Space Models
Developing architectures suitable for modeling raw audio is a challengin...
research
∙
10/31/2021
Efficiently Modeling Long Sequences with Structured State Spaces
A central goal of sequence modeling is designing a single principled mod...
research
∙
10/26/2021
Combining Recurrent, Convolutional, and Continuous-time Models with Linear State-Space Layers
Recurrent neural networks (RNNs), temporal convolutions, and neural diff...
research
∙
06/07/2021
HoroPCA: Hyperbolic Dimensionality Reduction via Horospherical Projections
This paper studies Principal Component Analysis (PCA) for data lying in ...
research
∙
12/29/2020
Kaleidoscope: An Efficient, Learnable Representation For All Structured Linear Maps
Modern neural network architectures use structured linear transformation...
research
∙
11/25/2020
No Subclass Left Behind: Fine-Grained Robustness in Coarse-Grained Classification Problems
In real-world classification tasks, each class often comprises multiple ...
research
∙
10/01/2020
From Trees to Continuous Embeddings and Back: Hyperbolic Hierarchical Clustering
Similarity-based Hierarchical Clustering (HC) is a classical unsupervise...
research
∙
08/17/2020
HiPPO: Recurrent Memory with Optimal Polynomial Projections
A central problem in learning from sequential data is representing cumul...
research
∙
08/15/2020
Model Patching: Closing the Subgroup Performance Gap with Data Augmentation
Classifiers in machine learning are often brittle when deployed. Particu...
research
∙
10/22/2019
Improving the Gating Mechanism of Recurrent Neural Networks
Gating mechanisms are widely used in neural network models, where they a...
research
∙
07/19/2019
Sparse Recovery for Orthogonal Polynomial Transforms
In this paper we consider the following sparse recovery problem. We have...
research
∙
03/14/2019
Learning Fast Algorithms for Linear Transforms Using Butterfly Factorizations
Fast linear transforms are ubiquitous in machine learning, including the...
research
∙
10/04/2018
Learning Compressed Transforms with Low Displacement Rank
The low displacement rank (LDR) framework for structured matrices repres...
research
∙
04/10/2018
Representation Tradeoffs for Hyperbolic Embeddings
Hyperbolic embeddings offer excellent quality with few dimensions when e...
research
∙
03/16/2018