1. Basics of Neuromorphic Design Sylvain Saïghi IMS Lab – University of Bordeaux

2. 2 Outline  Integrated Circuit Design  Models for neuromorphic implementation  Main types of neuromorphic systems GDR BioComp – Colloque 2015 – St Paul de Vence

3. 3 How to design an Integrated Circuit?  IC technology providers for prototyping and low volume production  Circuits Multi-Projets http://cmp.imag.fr/ http://cmp.imag.fr/  Europractice http://www.europractice-ic.com/ http://www.europractice-ic.com/  Same kind of providers in North America GDR BioComp – Colloque 2015 – St Paul de Vence

4. 4 How to design an Integrated Circuit?  Schedule GDR BioComp – Colloque 2015 – St Paul de Vence

5. 5 How to design an Integrated Circuit?  Schedule and Prices… GDR BioComp – Colloque 2015 – St Paul de Vence

6. 6 How to design an Integrated Circuit?  Second step: the schematic  Design GDR BioComp – Colloque 2015 – St Paul de Vence

7. 7 How to design an Integrated Circuit?  Second step: the schematic  Design  Simulation  Monte Carlo to check the reliability against the parameters mismatch GDR BioComp – Colloque 2015 – St Paul de Vence

8. 8 How to design an Integrated Circuit?  Third step: the layout  Mask drawings are sent to the IC provider  The provider gathers all chips from different designers GDR BioComp – Colloque 2015 – St Paul de Vence

9. 9 How to design an Integrated Circuit?  Third step: the layout  Mask drawings are sent to the IC provider  The provider gathers all chips from different designers  You will receive 25 samples 3 or 4 months later  Total duration about 12-18 months GDR BioComp – Colloque 2015 – St Paul de Vence

10. 10 Another solution for digital design  VHDL: Very High-Level Design Language  Descriptive language  Behavioral simulation  Choice of target IC (Founder library) FPGA (Field Programmable Gate Array)  Physical simulations GDR BioComp – Colloque 2015 – St Paul de Vence

11. 11 A neuron for neuromorphic designers GDR BioComp – Colloque 2015 – St Paul de Vence  A few figures  Soma diameter ~ 4 to 100  m  Axonal time propagation ~ 1 to 30ms  Membrane voltage from ~ -120mV to 50mV  Ionic currents ~ 10 nA

12. 12 A neuron for neuromorphic designers GDR BioComp – Colloque 2015 – St Paul de Vence  Spiking Neuron → time dependent SYNAPTIC CURRENTS

13. 13  Hebbian rule: Spike Timing Dependent Plasticity (STDP) Learning in Neuromorphic Systems NjNj NiNi Presynaptic neuron Postsynaptic neuron W ji Causality Increasing of Synaptic Weight (potentiation LTP) No causality Decreasing of Synaptic Weight (depression LTD)

14. 14 Threshold = 2 1/3

15. 15 2/3 Threshold = 2 2/3

16. 16 1 Threshold = 2 1 1 1/3

17. 17 1 Threshold = 2 1 2/3 0 0 0

18. 18 1 Threshold = 2 1 0 0 0 1/3

19. 19 1 Threshold = 2 1 0 0 0 0

20. 20 First neuromorphic design  Carver Mead – Caltech  The term neuromorphic was coined by Carver Mead, in the late 1980s to describe Very-Large-Scale Integration (VLSI) systems containing electronic circuits that emulate the bio-physics of neural systems using the physics of silicon. GDR BioComp – Colloque 2015 – St Paul de Vence 1989

21. 21 2 main types of neuromorphic design GDR BioComp – Colloque 2015 – St Paul de Vence DigitalAnalog Neuromimetic Rebuild, understand the life Biological Real Time (99% of case) Could be connected to life cells Bio-Inspired Engineering issues Event based computation ? ? ? ?

22. 22 Neuromimetic design NeuroGrid (Standford)  Simulate the brain in real-time  Specifications:  sixteen Neurocores each of which has 256 x 256 silicon neurons in an 11.9 mm x 13.9 mm chip  Analog core and Digital communication bus GDR BioComp – Colloque 2015 – St Paul de Vence “Neurogrid simulates a million neurons connected by billions of synapses in real-time, rivaling a supercomputer while consuming a 100,000 times less energy—five watts instead of a megawatt!”

23. 23 Neuromimetic design SpiNNaker (Manchester University) – Human Brain Project  Simulate the brain in real time  Specifications  a million-core computing engine  massively-parallel  fully digital design GDR BioComp – Colloque 2015 – St Paul de Vence

24. 24 Neuromimetic design Heidelberg’s system (Heidelberg University) – Human Brain Project  Simulate the brain 10 4 faster than real-time  Specifications  Based around wafer-scale VLSI  Analog core and Digital communication bus  Each 20-cm-diameter silicon wafer contains 384 chips, each of which implements 128,000 synapses and up to 512 spiking neurons. Total of around 200,000 neurons and 49 million synapses per wafer. GDR BioComp – Colloque 2015 – St Paul de Vence

25. 25 Bio-Inspired design  TrueNorth – IBM  Video treatment  Specifications  Fully digital GDR BioComp – Colloque 2015 – St Paul de Vence

26. 26 Bio-Inspired design  TrueNorth – IBM TrueNorth video GDR BioComp – Colloque 2015 – St Paul de Vence

27. 27 Bio-Inspired design  Zeroth – Qualcomm  Neural Processing Units which learn your abs  Specifications  Fully digital GDR BioComp – Colloque 2015 – St Paul de Vence

28. 28GDR BioComp – Colloque 2015 – St Paul de Vence Qualcomm video

29. 29 Other neuromorphic research groups (non-exhaustive)  Europe Institute of Neuroinformatics (Zurich) Instituto Microelectronica (Sevilla) University of Bordeaux …  North America Georgia Tech (Atlanta) Johns Hopkins University (Baltimore) University of California (San Diego) …  Asia University of New South Wales (Sydney) … GDR BioComp – Colloque 2015 – St Paul de Vence

30. Thanks for your attention