DSA patterning options for logics and memory applications
Chi Chun Liu, Elliott Franke, et al.
SPIE Advanced Lithography 2017
Recent advances in deep learning have been driven by ever-increasing model sizes, with networks growing to millions or even billions of parameters. Such enormous models call for fast and energy-efficient hardware accelerators. We study the potential of Analog AI accelerators based on Non-Volatile Memory, in particular Phase Change Memory (PCM), for software-equivalent accurate inference of natural language processing applications. We demonstrate a path to software-equivalent accuracy for the GLUE benchmark on BERT (Bidirectional Encoder Representations from Transformers), by combining noise-aware training to combat inherent PCM drift and noise sources, together with reduced-precision digital attention-block computation down to INT6.
Chi Chun Liu, Elliott Franke, et al.
SPIE Advanced Lithography 2017
Hsinyu Tsai, Stefano Ambrogio, et al.
VLSI Technology 2019
Adam Pyzyna, Hsinyu Tsai, et al.
IITC 2017
Sanjay Kariyappa, Hsinyu Tsai, et al.
IEEE T-ED