1. Hybrid Pipeline Structure for Self-Organizing Learning Array Yinyin Liu 1, Ding Mingwei 2, Janusz A. Starzyk 1, 1 School of Electrical Engineering & Computer Science Ohio University, USA 2 Ross University ISNN 2007: The 4th International Symposium on Neural Networks

2. 2 Outline RC systems design of SOLAR Dimensionality reduction Input selection, weighting Pipeline structure Experimental results Conclusions Broca’s area Pars opercularis Motor cortex Somatosensory cortex Sensory associative cortex Primary Auditory cortex Wernicke’s area Visual associative cortex Visual cortex

3. 3 “… Perhaps the last frontier of science – its ultimate challenge- is to understand the biological basis of consciousness and the mental process by which we perceive, act, learn and remember.. ” from Principles of Neural Science by E. R. Kandel et al. E. R. Kandel won Nobel Price in 2000 for his work on physiological basis of memory storage in neurons. “… The question of intelligence is the last great terrestrial frontier of science... ” from Jeff Hawkins On Intelligence. Jeff Hawkins founded the Redwood Neuroscience Institute devoted to brain research. He co-founded Palm Computing and Handspring Inc. Intelligence AI’s holy grail From Pattie Maes MIT Media Lab

4. 4 How can we design intelligence? We need to know how We need means to implement it We need resources to build and sustain its operation

5. 5 From Ray Kurzwail, The Singularity Summit at Stanford, May 13, 2006 Resources – Evolution of Electronics

6. 6 By Gordon E. Moore

7. 7

8. 8 From Ray Kurzwail, The Singularity Summit at Stanford, May 13, 2006 Clock Speed (doubles every 2.7 years)

9. 9 From Ray Kurzwail, The Singularity Summit at Stanford, May 13, 2006

10. 10 Outline RC systems design of SOLAR Dimensionality reduction Input selection, weighting Pipeline structure Experimental results Conclusions Broca’s area Pars opercularis Motor cortex Somatosensory cortex Sensory associative cortex Primary Auditory cortex Wernicke’s area Visual associative cortex Visual cortex

11. 11 Traditional ANN Hardware –Limited routing resource. –Quadratic relationship between the routing and the number of neuron makes classical ANNs wire dominated. input output information flow hidden Interconnect is 70% of chip area

12. 12 Biological Neural Networks Biological Neural Networks Cell body From IFC’s webpage Dowling, 1998, p. 17

13. 13 Sparse Structure 10 12 neurons in human brain are sparsely connected On average, each neuron is connected to other neurons through about 10 4 synapses Sparse structure enables efficient computation and saves energy and cost

14. 14 Why should we care? Source: SEMATECH

15. 15 Percent of die area that must be occupied by memory to maintain SOC design productivity Design Productivity Gap  Low-Value Designs? Source = Japanese system-LSI industry

16. 16 Outline RC systems design of SOLAR Dimensionality reduction Input selection, weighting Pipeline structure Experimental results Conclusions Broca’s area Pars opercularis Motor cortex Somatosensory cortex Sensory associative cortex Primary Auditory cortex Wernicke’s area Visual associative cortex Visual cortex

17. 17 SOLAR System Design SOLAR Introduction Entropy based self- organization – data-driven – Local connection Dynamical reconfiguration Local and sparse i nterconnections Online inputs selection Feature neurons and merging neurons Pattern recognition, classification

18. 18 Pipeline Overview node computing ability → “soft” connections Four modes 1. Idle 2. Read 3. Process 4. Write

19. 19 Pipeline Signal Flow 1

20. 20 Pipeline Signal Flow 2

21. 21 Pipeline Signal Flow 3

22. 22 Node Operations Implemented with Xilinx picoBlaze Runs at higher frequency

23. 23 Outline RC systems design of SOLAR Dimensionality reduction Input selection, weighting Pipeline structure Experimental resultsExperimental results Conclusions Broca’s area Pars opercularis Motor cortex Somatosensory cortex Sensory associative cortex Primary Auditory cortex Wernicke’s area Visual associative cortex Visual cortex

24. 24 Em(x) Simulation Results

25. 25 Iris Data Processing 4x7 array processing Iris data Linear growth of HW cost

26. 26 Chip Layout

27. 27 XILINX VIRTEX XCV 1000 Hardware Development

28. 28 Future Work - System SOLAR

29. 29 Conclusions & Future work Sparse coding building in sparsely connected networks WTA scheme: local competition accomplish the global competition using primary and secondary layers –efficient hardware implementation OTA scheme: local competition produces neuronal activity reduction OTA – redundant coding: more reliable and robust WTA & OTA: learning memory for developing machine intelligence Future work: Introducing temporal sequence learning Building motor pathway on such learning memory Combining with goal-creation pathway to build intelligent machine