Enhancing Unsupervised Speech Recognition with Diffusion GANs

03/23/2023

∙

by   Xianchao Wu, et al.

∙

0

∙

share

We enhance the vanilla adversarial training method for unsupervised Automatic Speech Recognition (ASR) by a diffusion-GAN. Our model (1) injects instance noises of various intensities to the generator's output and unlabeled reference text which are sampled from pretrained phoneme language models with a length constraint, (2) asks diffusion timestep-dependent discriminators to separate them, and (3) back-propagates the gradients to update the generator. Word/phoneme error rate comparisons with wav2vec-U under Librispeech (3.1 test-clean and 5.6 enhancement strategies work effectively.