Hardware accelerator for training with integer backpropagation and probabilistic weight update

Advances in the architecture of inference accelerators and quantization techniques of neural networks allow effective on-device inference in embedded devices. Privacy issues for user data, as well as increasing needs of user-specific services, have led to a need for on-device training. The dot product operation required in backpropagation can be computed efficiently by multiplier–accumulators (MACs) in the inference accelerator if forward and backward propagation of the neural network have the same precision. This chapter introduces a quantization technique to enable computation by the digital neuron inference accelerator with the same precision as that using the forward path. Updating the 5-bit weights with gradients of higher precision is challenging. To address this issue, this chapter also introduces a probabilistic weight update. It also describes the hardware implementation of the probabilistic weight-update scheme. The proposed training technique achieves 98.15% recognition accuracy on the MNIST dataset.