Impact of PCM noise on the Spiking Restricted Boltzmann Machine via On-Chip Trainable PCM synapses

Abstract

The spiking neural network (SNN) has been highlighted by its brain-like computing based on spike activities which could bring power-efficient performance, further suggested as the next generation of neural networks [1]. However, hardware implementation of SNN is still challenging as to be designed friendly for both hardware and algorithm, to process large amounts of spatiotemporal spike trains efficiently. In this work, we introduce our recent results [2] based on a fully functional chip with a sizable number of 1.4M synaptic phase-change memory (PCM) cells and all the Si CMOS circuits needed for the targeting algorithm, spiking restricted Boltzmann machine (RBM). To implement the spiking RBM and its training algorithm, event-driven contrastive divergence (eCD) [3], synaptic crossbar unit cells are constructed with 6-transistor/2-PCM-resistor (6T2R) and peripheral stochastic leaky-integrate-and-fire (LIF) neuron circuits on fully silicon-integrated 90nm CMOS technology. A bidirectional, asynchronous, and parallel pulse-signaling scheme over an analog-weighted PCM synapse array is elaborately designed to be fitted with spike-timing-dependent plasticity local learning rule based on eCD. Since the RBM is a two-layer bidirectional neural network with a probabilistic-based algorithm, we elucidate the system basis with on-chip characterization results such as bidirectional LIF operations over various weighted PCM cells and sigmoid-like firing probability via random walk circuitry. A reasonable symmetricity between LIF and BLIF as well as the stochasticity from on-chip stochastic neuron circuits are shown, which are of critical importance to execute the Spiking RBM. The densely integrated spiking RBM chip is configured with a field- programmable gate array (FPGA)-based evaluation system to efficiently transfer spike trains through on-chip scan chains to operate RBM algorithm phases. A fully hardware demonstration of pattern inference scored 93% on-chip training accuracy from 100 MNIST digit handwritten image samples. Furthermore, to clarify the generative performances of the spiking RBM chip, we experimentally demonstrated image reconstruction by inputting partial unperfect patterns. Simulation studies are also added to evaluate how well the impact of PCM- based intrinsic 1/f noise is suitable for the generative characteristic of spiking RBM. Thanks to well-fabricated units such as neuron circuits and synaptic cells operating asynchronously and parallelly, this work indicates the potential of power-efficient SNN processors by taking inherent advantage of spike sparseness. *U. Shin and M. Ishii contributed equally to this work. [1] W. Maass, Neural Networks, 10, 1659, (1997). [2] U. Shin, M. Ishii et al., Adv. Intell. Syst. 2200034, (2022). [3] E. Neftci et al., Front. Neurosci, 7, 272, (2014).