research
∙
08/10/2023
EXPRESSO: A Benchmark and Analysis of Discrete Expressive Speech Resynthesis
Recent work has shown that it is possible to resynthesize high-quality s...
share
research
∙
06/23/2023
Voicebox: Text-Guided Multilingual Universal Speech Generation at Scale
Large-scale generative models such as GPT and DALL-E have revolutionized...
research
∙
05/22/2023
Scaling Speech Technology to 1,000+ Languages
Expanding the language coverage of speech technology has the potential t...
research
∙
05/17/2023
DinoSR: Self-Distillation and Online Clustering for Self-supervised Speech Representation Learning
In this paper, we introduce self-distillation and online clustering for ...
research
∙
03/20/2023
Cocktail HuBERT: Generalized Self-Supervised Pre-training for Mixture and Single-Source Speech
Self-supervised learning leverages unlabeled data effectively, improving...
research
∙
03/01/2023
MuAViC: A Multilingual Audio-Visual Corpus for Robust Speech Recognition and Robust Speech-to-Text Translation
We introduce MuAViC, a multilingual audio-visual corpus for robust speec...
research
∙
02/10/2023
AV-data2vec: Self-supervised Learning of Audio-Visual Speech Representations with Contextualized Target Representations
Self-supervision has shown great potential for audio-visual speech recog...
research
∙
01/10/2023
Scaling Laws for Generative Mixed-Modal Language Models
Generative language models define distributions over sequences of tokens...
research
∙
12/21/2022
ReVISE: Self-Supervised Speech Resynthesis with Visual Input for Universal and Generalized Speech Enhancement
Prior works on improving speech quality with visual input typically stud...
research
∙
12/14/2022
Efficient Self-supervised Learning with Contextualized Target Representations for Vision, Speech and Language
Current self-supervised learning algorithms are often modality-specific ...
research
∙
12/02/2022
Continual Learning for On-Device Speech Recognition using Disentangled Conformers
Automatic speech recognition research focuses on training and evaluating...
research
∙
11/11/2022
Speech-to-Speech Translation For A Real-world Unwritten Language
We study speech-to-speech translation (S2ST) that translates speech from...
research
∙
10/18/2022
Simple and Effective Unsupervised Speech Translation
The amount of labeled data to train models for speech tasks is limited f...
research
∙
07/14/2022
A Single Self-Supervised Model for Many Speech Modalities Enables Zero-Shot Modality Transfer
While audio-visual speech models can yield superior performance and robu...
research
∙
05/15/2022
Learning Lip-Based Audio-Visual Speaker Embeddings with AV-HuBERT
This paper investigates self-supervised pre-training for audio-visual sp...
research
∙
04/25/2022
On-demand compute reduction with stochastic wav2vec 2.0
Squeeze and Efficient Wav2vec (SEW) is a recently proposed architecture ...
research
∙
04/11/2022
Unified Speech-Text Pre-training for Speech Translation and Recognition
We describe a method to jointly pre-train speech and text in an encoder-...
research
∙
04/06/2022
Enhanced Direct Speech-to-Speech Translation Using Self-supervised Pre-training and Data Augmentation
Direct speech-to-speech translation (S2ST) models suffer from data scarc...
research
∙
04/06/2022
Simple and Effective Unsupervised Speech Synthesis
We introduce the first unsupervised speech synthesis system based on a s...
research
∙
04/05/2022
Towards End-to-end Unsupervised Speech Recognition
Unsupervised speech recognition has shown great potential to make Automa...
research
∙
03/30/2022
Generative Spoken Dialogue Language Modeling
We introduce dGSLM, the first "textless" model able to generate audio sa...
research
∙
03/01/2022
Measuring the Impact of Individual Domain Factors in Self-Supervised Pre-Training
Human speech data comprises a rich set of domain factors such as accent,...
research
∙
02/15/2022
textless-lib: a Library for Textless Spoken Language Processing
Textless spoken language processing research aims to extend the applicab...
research
∙
02/07/2022
data2vec: A General Framework for Self-supervised Learning in Speech, Vision and Language
While the general idea of self-supervised learning is identical across m...
research
∙
01/05/2022
Robust Self-Supervised Audio-Visual Speech Recognition
Audio-based automatic speech recognition (ASR) degrades significantly in...
research
∙
01/05/2022
Learning Audio-Visual Speech Representation by Masked Multimodal Cluster Prediction
Video recordings of speech contain correlated audio and visual informati...
research
∙
12/15/2021
Textless Speech-to-Speech Translation on Real Data
We present a textless speech-to-speech translation (S2ST) system that ca...
research
∙
11/14/2021
Textless Speech Emotion Conversion using Decomposed and Discrete Representations
Speech emotion conversion is the task of modifying the perceived emotion...
research
∙
10/15/2021
Direct simultaneous speech to speech translation
We present the first direct simultaneous speech-to-speech translation (S...
research
∙
09/14/2021
fairseq S^2: A Scalable and Integrable Speech Synthesis Toolkit
This paper presents fairseq S^2, a fairseq extension for speech synthesi...
research
∙
09/07/2021
Text-Free Prosody-Aware Generative Spoken Language Modeling
Speech pre-training has primarily demonstrated efficacy on classificatio...
research
∙
07/12/2021
Direct speech-to-speech translation with discrete units
We present a direct speech-to-speech translation (S2ST) model that trans...
research
∙
06/14/2021
Kaizen: Continuously improving teacher using Exponential Moving Average for semi-supervised speech recognition
In this paper, we introduce the Kaizen framework that uses a continuousl...
research
∙
06/14/2021
HuBERT: Self-Supervised Speech Representation Learning by Masked Prediction of Hidden Units
Self-supervised approaches for speech representation learning are challe...
research
∙
05/24/2021
Unsupervised Speech Recognition
Despite rapid progress in the recent past, current speech recognition sy...
research
∙
04/02/2021
Robust wav2vec 2.0: Analyzing Domain Shift in Self-Supervised Pre-Training
Self-supervised learning of speech representations has been a very activ...
research
∙
04/01/2021
Speech Resynthesis from Discrete Disentangled Self-Supervised Representations
We propose using self-supervised discrete representations for the task o...
research
∙
02/01/2021
Generative Spoken Language Modeling from Raw Audio
Generative spoken language modeling involves learning jointly the acoust...
research
∙
12/31/2020
Text-Free Image-to-Speech Synthesis Using Learned Segmental Units
In this paper we present the first model for directly synthesizing fluen...
research
∙
10/02/2020
Differentiable Weighted Finite-State Transducers
We introduce a framework for automatic differentiation with weighted fin...
research
∙
06/03/2020
A Convolutional Deep Markov Model for Unsupervised Speech Representation Learning
Probabilistic Latent Variable Models (LVMs) provide an alternative to se...
research
∙
02/24/2020
Semi-Supervised Speech Recognition via Local Prior Matching
For sequence transduction tasks like speech recognition, a strong struct...
research
∙
11/21/2019
Learning Hierarchical Discrete Linguistic Units from Visually-Grounded Speech
In this paper, we present a method for learning discrete linguistic unit...
research
∙
07/09/2019
Transfer Learning from Audio-Visual Grounding to Speech Recognition
Transfer learning aims to reduce the amount of data required to excel at...
research
∙
04/05/2019
An Unsupervised Autoregressive Model for Speech Representation Learning
This paper proposes a novel unsupervised autoregressive neural model for...
research
∙
02/21/2019
Lingvo: a Modular and Scalable Framework for Sequence-to-Sequence Modeling
Lingvo is a Tensorflow framework offering a complete solution for collab...
research
∙
10/16/2018
Hierarchical Generative Modeling for Controllable Speech Synthesis
This paper proposes a neural end-to-end text-to-speech (TTS) model which...
research
∙
09/12/2018
Unsupervised Representation Learning of Speech for Dialect Identification
In this paper, we explore the use of a factorized hierarchical variation...
research
∙
08/30/2018
Semi-Supervised Training for Improving Data Efficiency in End-to-End Speech Synthesis
Although end-to-end text-to-speech (TTS) models such as Tacotron have sh...
research
∙
06/13/2018