EDA Challenges in Neuromorphic Computing Embrace the BRAIN Century: EDA Challenges in Neuromorphic Computing Hai Li and Yiran Chen Evolutionary Intelligence Lab (EI-Lab) Electrical and Computer Engineering University of Pittsburgh

What is Neuromorphic Computing? An interdisciplinary technology that was inspired from biology, physics, mathematics, computer science, and electronic engineering to design artificial neural systems. (Wikipedia) It is supposed to fulfill the weakness of von Neumann architecture in processing cognitive applications. The relevant research has been well funded by all major funding agencies: And supported in many countries:

Question I – Understanding? Unfortunately we still do not know much about human brains. The artificial neural network models also evolves over years. Representation of neuron: 1943, McCulloch (Pitt) The 1st learning rule: 1949, Hebb Neuron nets: 1955, Dartmouth Summer Research Project on AI STDP (Spike-timing-dependent plasticity): 1973, Taylor CNN (Convolutional neural networks): 1989, LeCun Do we really need to understand brains before designing a useful N.C. system? No. Many useful systems have been prototyped, e.g., IBM TrueNorth. The debates on “Emulative vs. Simulative”.

Application Specific IC Memristor Based Reconfigurable Design Question II – Platform? Application Specific IC Misra et al, Neurocomputing, 2010 Programmable Hardware Graf et al, NIPS, 2009 Misra et al, Neurocomputing, 2010 Adaptivity (AD) Performance (PE) Power Efficiency (PO) Programmability (PR) Scalability (SC) Memristor Based Reconfigurable Design H. Li, HPEC, 2010 4, DAC, 2015 General Purpose Platform P. J. Fox, Tech. Report, 2013 Graf et al, NIPS, 2009 AD: how easy it adapts various NN models, high AD means it can support new NN models without complete redesign of hardware. PR: user friendly, high PR means user can model the NN well without knowing the underlining hardware. SC: the scalability of particular platform, high SC means current platform is easily or capable of freely scaling the problem size of NN models. General purpose platform: easy programming, very flexible to support different NN models (adaptivity), can scale to large NNs with billions of neurons, but low performance and poor power efficiency. Programmable hardware: tradeoff the programmability, scalability and adaptivity for better performance and power efficiency, expensive to support large scale NNs given the low circuit density etc… ASIC: further sacrificing programmability, scalability and adaptivity can bring much more powerful acc. solutions, which gives us the choice of ASIC. Memristor based ASIC: digital or analog ASICs are still limited by the massive hardware connection between different layers in NNs. The memristor based acc. we propose has well scalability and breaks the limitation of traditional ASIC designs with architecture innovations. NN is popular enough today and it deserves a special hardware acc. solution, offsetting the reduced programmability compared to CPU/GPGPU/FPGA choices.

Question III – Technologies? Are conventional CMOS and EDA technologies capable to support long-term research and development of N.C. systems? Debates Analog or Digital? Spiking-based or level-based? Synchronous or asynchronous? CMOS or Post-Silicon? Other Challenges Programmability Reliability Scalability Security J. Gehlhaar, ASPLOS, 2014 J. Hsu, IEEE Spectrum, 2014 IBM, TrueNorth SRAM synapse Digital spike 1M neurons/chip 256M synapse/chip Qualcomm, Zeroth Custom hybrid Spike neurons on chip Synapse off chip B. Benjamin, Neurogrid, 2014 Stanford, Brain in Silicon Mixed-signal VLSI 1M neurons/16 chips 1B synapse/16 chips D.B. Strukov, Nature, 2014 HP, memristor X-bar Analog computing Dense connection F. Samarrai, UVAToday, 2014 Micron, Automata Massively parallel Memory driven Non-von Neumann XML-based language S. Miller, ESANN, 2012 HBP Analog VLSI 64 neurons/chip 1024 synapses/chip

Q & A? Acknowledgement Dr. Daniel Hammerstrom, Program manager, DARPA Dr. Robinson Pino, Program manager, DOE Dr. Dharmendra S. Modha, IBM Fellow and IBM Chief Scientist for Brain-inspired Computers Dr. Mark Barnell, Senior computer scientist and program manager, US AFRL Dr. H.-S. Philip Wong, Willard R. and Inez Kerr Bell Professor, Stanford University Q & A?