1. Overview of Neuromorphic Computing
Chris Carothers, CCI Director
Jim Hendler, IDEA Director
Rensselaer Polytechnic Institute

2. Outline Background AFRL: Hybrid Neuromorphic Supercomputers
Biological Neuron Structure
From Bio Neuron to Silicon Neuron
IBM TrueNorth Architecture
AFRL: Hybrid Neuromorphic Supercomputers
Future Research Areas

3. Structure of a Real Neuron (from KSJ 4e, 2001)
Most neurons in the vertebrate nervous system have several main features in common. The cell body contains the nucleus, the storehouse of genetic information, and gives rise to two types of cell processes, axons and dendrites. Axons, the transmitting element of neurons, can vary greatly in length; some can extend more than 3 m within the body. Most axons in the central nervous system are very thin (between 0.2 and 20 µm in diameter) compared with the diameter of the cell body (50 µm or more). Many axons are insulated by a fatty sheath of myelin that is interrupted at regular intervals by the nodes of Ranvier. The action potential, the cell's conducting signal, is initiated either at the axon hillock, the initial segment of the axon, or in some cases slightly farther down the axon at the first node of Ranvier. Branches of the axon of one neuron (the presynaptic neuron) transmit signals to another neuron (the postsynaptic cell) at a site called the synapse. The branches of a single axon may form synapses with as many as 1000 other neurons. Whereas the axon is the output element of the neuron, the dendrites (apical and basal) are input elements of the neuron. Together with the cell body, they receive synaptic contacts from other neurons.

4. Neuron Classification (from KSJ 4e, 2000)
Neurons can be classified as unipolar, bipolar, or multipolar according to the number of processes that originate from the cell body.
A. Unipolar cells have a single process, with different segments serving as receptive surfaces or releasing terminals. Unipolar cells are characteristic of the invertebrate nervous system.
B. Bipolar cells have two processes that are functionally specialized: the dendrite carries information to the cell, and the axon transmits information to other cells.
C. Certain neurons that carry sensory information, such as information about touch or stretch, to the spinal cord belong to a subclass of bipolar cells designated as pseudo-unipolar. As such cells develop, the two processes of the embryonic bipolar cell become fused and emerge from the cell body as a single process. This outgrowth then splits into two processes, both of which function as axons, one going to peripheral skin or muscle, the other going to the central spinal cord.
D. Multipolar cells have an axon and many dendrites. They are the most common type of neuron in the mammalian nervous system. Three examples illustrate the large diversity of these cells. Spinal motor neurons (left) innervate skeletal muscle fibers. Pyramidal cells (middle) have a roughly triangular cell body; dendrites emerge from both the apex (the apical dendrite) and the base (the basal dendrites). Pyramidal cells are found in the hippocampus and throughout the cerebral cortex. Purkinje cells of the cerebellum (right) are characterized by the rich and extensive dendritic tree in one plane. Such a structure permits enormous synaptic input.

5. From Bio Neurons to Silicon Neurons (Hasler/Marr ‘13)

6. IBM TrueNorth Architecture (Cassidy et al)
Neuromorphic core: 256 axons, 256 neurons and 256x256 synapses
Architecture rich enough to support both neural network/compute algorithms and neural biologically relevant behaviors

7. IBM TrueNorth (alt. view Cassidy et al )

8. IBM TrueNorth Neuron Model (Cassidy et al)
Note: Vj(t) is a 20 bit signed integer

9. IBM TrueNorth Lib & Programming (Cassidy et al and Amir et al)

10. TN’s 20 Biological Relevant Behaviors (Cassidy et al)

11. IBM TrueNorth Performance (Cassidy et al)
TN Peak performance is ~400G 200
Hasler’s scale indicates 1 Exa-SOPS/watt to reach brain efficiencies
Analog SP is at 1 Tera-SOPS/watt

12. Other Related Projects
Human Brain Project
SpiNNaker – Manchester UK, 18 ARM chips, 250K neurons & 80M synapes in 36 watts.
Blue Brain Project – EPFL, models all the details of the human brain, simulation runs on BG/Q at JSC in Germany
Qualcomm Zeroth NPU
See Jeff Gehlhaar’s ASPLOS ‘14 keynote
Similar goals as TrueNorth, few specs available
Neurogrid – Stanford/Cornell
Analog neuron
1M neurons & ~1B synaspe in 5 watts
Funded by NSF EMT program
HP’s Neuristor
Specialized transistor that emulates neuron functionality more directly.

13. AFRL Proposal: Motivation
Significant shift from parallelism across nodes to parallelism within nodes (very “fat” nodes)
Intra-node parallelism most exploit some sort of “streaming/vector” GPU-like processing
Potential to leave behind “asynchronous” data/compute apps
Fault tolerance & power (data movement) is a big challenge..
Virtual Data Facility is a COORDINATED, MULTI-SITE FACILITY whose purpose is to address the SHARED DATA INFRASTRUCTURE NEEDS of the Office of Science.
ESNet == Energy Sciences Network!!

14. AFRL Project: Hybrid Neuromorphic Supercomputer
The question: how might a neuromorphic “accelerator” type processor be used to improve the application performance, power consumption and overall system reliability of future exascale systems?
Driven by the recent DOE SEAB report on high-performance computing [22] which high-lights the neuromorphic architecture as one that “is an emergent area for exploitation”.

15. Future Research Directions
Alt. Neuromorphic Architectures
Increase degree of asynchronous spike event processing
Increase core size, e.g., 1K, 4K or 8K neurons & axons
Multi-cycle neurons and not single-cycle clock
Mix of heterogeneous neuron cores
Dataflow and not time-flow driven approach
High “burst” clock rate? E.g., 1GHz neurons
Application areas:
Neuromorphic cybersecurity
Neuromorphic OS/run-time system
Fault tolerance is only one important function…
Neuromorphic Data Mining
Neuromorphic (Sensor) Network
Bio related apps (e.g., ultra fast MD sim  Anton)