DeepAI AI Chat
Log In Sign Up

Spiking-Diffusion: Vector Quantized Discrete Diffusion Model with Spiking Neural Networks

by   Mingxuan Liu, et al.

Spiking neural networks (SNNs) have tremendous potential for energy-efficient neuromorphic chips due to their binary and event-driven architecture. SNNs have been primarily used in classification tasks, but limited exploration on image generation tasks. To fill the gap, we propose a Spiking-Diffusion model, which is based on the vector quantized discrete diffusion model. First, we develop a vector quantized variational autoencoder with SNNs (VQ-SVAE) to learn a discrete latent space for images. In VQ-SVAE, image features are encoded using both the spike firing rate and postsynaptic potential, and an adaptive spike generator is designed to restore embedding features in the form of spike trains. Next, we perform absorbing state diffusion in the discrete latent space and construct a spiking diffusion image decoder (SDID) with SNNs to denoise the image. Our work is the first to build the diffusion model entirely from SNN layers. Experimental results on MNIST, FMNIST, KMNIST, Letters, and Cifar10 demonstrate that Spiking-Diffusion outperforms the existing SNN-based generation model. We achieve FIDs of 37.50, 91.98, 59.23, 67.41, and 120.5 on the above datasets respectively, with reductions of 58.60%, 18.75%, 64.51%, 29.75%, and 44.88% in FIDs compared with the state-of-art work. Our code will be available at <>.


page 4

page 8


Fully Spiking Variational Autoencoder

Spiking neural networks (SNNs) can be run on neuromorphic devices with u...

Spiking Denoising Diffusion Probabilistic Models

Spiking neural networks (SNNs) have ultra-low energy consumption and hig...

Navigating Local Minima in Quantized Spiking Neural Networks

Spiking and Quantized Neural Networks (NNs) are becoming exceedingly imp...

Inherent Redundancy in Spiking Neural Networks

Spiking Neural Networks (SNNs) are well known as a promising energy-effi...

The fine line between dead neurons and sparsity in binarized spiking neural networks

Spiking neural networks can compensate for quantization error by encodin...

Deep Compressive Autoencoder for Action Potential Compression in Large-Scale Neural Recording

Understanding the coordinated activity underlying brain computations req...

Membrane Potential Batch Normalization for Spiking Neural Networks

As one of the energy-efficient alternatives of conventional neural netwo...