Weight Programming in DNN Analog Hardware Accelerators in the Presence of NVM Variability

Abstract

Crossbar arrays of nonvolatile memory (NVM) can potentially accelerate development of deep neural networks (DNNs) by implementing crucial multiply–accumulate (MAC) operations at the location of data. Effective weight-programming procedures can both minimize the performance impact during training and reduce the down time for inference, where new parameter sets may need to be loaded. Simultaneous weight programming along an entire dimension (e.g., row or column) of a crossbar array in the context of forward inference and training is shown to be important. A framework for determining the optimal hardware conditions in which to program weights is provided, and its efficacy in the presence of considerable NVM variability is explored through simulations. This strategy is shown capable of programming 98–99% of weights effectively, in a manner that is both largely independent of the target weight distribution and highly tolerant to variability in NVM conductance-versus-pulse characteristics. The probability that a weight fails to reach its target value, termed $ P_{fail} $ , is quantified and the fundamental trade-off between $ P_{fail} $ and weight programming speed is explored. Lastly, the impact of imperfectly programmed weights on DNN test accuracies is examined for various networks, including multi-layer perceptrons (MLPs) and long short-term memory (LSTM).