Surrogate-gradient training of SNNs

Spiking neurons emit discrete, non-differentiable spikes, so ordinary backpropagation cannot flow gradients through the threshold. Surrogate-gradient methods replace the spike's zero-or-infinite derivative with a smooth surrogate (for example a fast sigmoid or triangular function of the membrane potential) during the backward pass only, while the forward pass still spikes. This makes an SNN trainable by backpropagation-through-time like a recurrent network, and is currently the most effective way to train accurate spiking decoders from scratch.

Choices of surrogate shape, time discretisation, and how to handle temporal credit assignment affect accuracy and cost, and training remains more delicate and memory-hungry than for standard networks. It is the main alternative to converting a pre-trained analog network into spikes.