Hybrid Pipeline Structure for Self-Organizing Learning Array Yinyin Liu 1, Ding Mingwei 2, Janusz A. Starzyk 1, 1 School of Electrical Engineering & Computer.

Slides:



Advertisements
Similar presentations
Introduction to Neural Networks
Advertisements

Bioinspired Computing Lecture 14
20031 Janusz Starzyk and Yongtao Guo School of Electrical Engineering and Computer Science Ohio University, Athens, OH 45701, U.S.A. September,
Sparse Coding in Sparse Winner networks Janusz A. Starzyk 1, Yinyin Liu 1, David Vogel 2 1 School of Electrical Engineering & Computer Science Ohio University,
Template design only ©copyright 2008 Ohio UniversityMedia Production Spring Quarter  A hierarchical neural network structure for text learning.
The Human Brain.
Analog Circuits for Self-organizing Neural Networks Based on Mutual Information Janusz Starzyk and Jing Liang School of Electrical Engineering and Computer.
Introduction: Neurons and the Problem of Neural Coding Laboratory of Computational Neuroscience, LCN, CH 1015 Lausanne Swiss Federal Institute of Technology.
Functional Link Network. Support Vector Machines.
[1].Edward G. Jones, Microcolumns in the Cerebral Cortex, Proc. of National Academy of Science of United States of America, vol. 97(10), 2000, pp
Brain-like design of sensory-motor programs for robots G. Palm, Uni-Ulm.
Self-Organizing Hierarchical Neural Network
EE141 EE690 Design of Embodied Intelligence Janusz Starzyk EECS, Ohio University.
EE141 1 Design of Self-Organizing Learning Array for Intelligent Machines Janusz Starzyk School of Electrical Engineering and Computer Science Heidi Meeting.
1 OPTIMIZED INTERCONNECTIONS IN PROBABILISTIC SELF-ORGANIZING LEARNING Janusz Starzyk, Mingwei Ding, Haibo He School of EECS Ohio University, Athens, OH.
Un Supervised Learning & Self Organizing Maps Learning From Examples
Emergence of Machine Intelligence. “…Perhaps the last frontier of science – its ultimate challenge- is to understand the biological basis of consciousness.
EE141 Challenges of Embodied Intelligence Janusz Starzyk, Yinyin Liu, Haibo He School of Electrical Engineering and Computer Science, Ohio University,
Fourth International Symposium on Neural Networks (ISNN) June 3-7, 2007, Nanjing, China A Hierarchical Self-organizing Associative Memory for Machine Learning.
Future Hardware Realization of Self-Organizing Learning Array and Its Software Simulation Adviser: Dr. Janusz Starzyk Student: Tsun-Ho Liu Ohio University.
EE141 1 Broca’s area Pars opercularis Motor cortexSomatosensory cortex Sensory associative cortex Primary Auditory cortex Wernicke’s area Visual associative.
November 5, 2009Introduction to Cognitive Science Lecture 16: Symbolic vs. Connectionist AI 1 Symbolism vs. Connectionism There is another major division.
EE141 1 Design of Self-Organizing Learning Array for Intelligent Machines Janusz Starzyk School of Electrical Engineering and Computer Science Heidi Meeting.
1 FPGA Lab School of Electrical Engineering and Computer Science Ohio University, Athens, OH 45701, U.S.A. An Entropy-based Learning Hardware Organization.
Associative Learning in Hierarchical Self Organizing Learning Arrays Janusz A. Starzyk, Zhen Zhu, and Yue Li School of Electrical Engineering and Computer.
On Intelligence Jeff Hawkins –Founder, Palm Computing: Palm Pilot –Founder, Handspring: Treo –Founder: Numenta Redwood Neuroscience Institute Redwood.
Fourth International Symposium on Neural Networks (ISNN) June 3-7, 2007, Nanjing, China Online Dynamic Value System for Machine Learning Haibo He, Stevens.
ICCINC' Janusz Starzyk, Yongtao Guo and Zhineng Zhu Ohio University, Athens, OH 45701, U.S.A. 6 th International Conference on Computational.
DESIGN OF A SELF- ORGANIZING LEARNING ARRAY SYSTEM Dr. Janusz Starzyk Tsun-Ho Liu Ohio University School of Electrical Engineering and Computer Science.
EE141 How to Motivate Machines to Learn and Help Humans in Making Water Decisions? Janusz Starzyk School of Electrical Engineering and Computer Science,
Self-Organized Recurrent Neural Learning for Language Processing April 1, March 31, 2012 State from June 2009.
Self organizing maps 1 iCSC2014, Juan López González, University of Oviedo Self organizing maps A visualization technique with data dimension reduction.
Advances in Modeling Neocortex and its impact on machine intelligence Jeff Hawkins Numenta Inc. VS265 Neural Computation December 2, 2010 Documentation.
Artificial Neural Networks. Applied Problems: Image, Sound, and Pattern recognition Decision making  Knowledge discovery  Context-Dependent Analysis.
Neural Network with Memory and Cognitive Functions Janusz A. Starzyk, and Yue Li School of Electrical Engineering and Computer Science Ohio University,
Cognition, Brain and Consciousness: An Introduction to Cognitive Neuroscience Edited by Bernard J. Baars and Nicole M. Gage 2007 Academic Press Chapter.
Page 1 Reconfigurable Communications Processor Principal Investigator: Chris Papachristou Task Number: NAG Electrical Engineering & Computer Science.
EE141 Motivated Learning based on Goal Creation Janusz Starzyk School of Electrical Engineering and Computer Science, Ohio University, USA
MAPLD 2005/254C. Papachristou 1 Reconfigurable and Evolvable Hardware Fabric Chris Papachristou, Frank Wolff Robert Ewing Electrical Engineering & Computer.
Language Project.  Neural networks have a large appeal to many researchers due to their great closeness to the structure of the brain, a characteristic.
Paper Review Presentation Paper Title: Hardware Assisted Two Dimensional Ultra Fast Placement Presented by: Mahdi Elghazali Course: Reconfigurable Computing.
Cognitive Science Overview Cognitive Science Defined The Brain Assumptions of Cognitive Science Cognitive Information Processing Cognitive Science and.
Hirota lab. 1 Mentality Expression by the eyes of a Robot Presented by: Pujan Ziaie Supervisor: Prof. K. Hirota Dept. of Computational Intelligence and.
Artificial Intelligence & Neural Network
4/19/20021 TCPSplitter: A Reconfigurable Hardware Based TCP Flow Monitor David V. Schuehler.
Speech Communication Lab, State University of New York at Binghamton Dimensionality Reduction Methods for HMM Phonetic Recognition Hongbing Hu, Stephen.
Andrew Ng, Director, Stanford Artificial Intelligence Lab
Lecture 5 Neural Control
Why Can't A Computer Be More Like A Brain?. Outline Introduction Turning Test HTM ◦ A. Theory ◦ B. Applications & Limits Conclusion.
20031 Janusz Starzyk, Yongtao Guo and Zhineng Zhu Ohio University, Athens, OH 45701, U.S.A. April 27 th, 2003.
Ghent University Pattern recognition with CNNs as reservoirs David Verstraeten 1 – Samuel Xavier de Souza 2 – Benjamin Schrauwen 1 Johan Suykens 2 – Dirk.
Cognitive Science Overview Introduction, Syllabus
Ghent University Compact hardware for real-time speech recognition using a Liquid State Machine Benjamin Schrauwen – Michiel D’Haene David Verstraeten.
1 Andrew Ng, Associate Professor of Computer Science Robots and Brains.
Brain, Mind, and Computation Part I: Computational Brain Brain, Mind, and Computation Part I: Computational Brain Brain-Mind-Behavior Seminar May 18, 2011.
Vocab 3b The Brain. area at the front of the parietal lobes that registers and processes body touch and movement sensations.
March 31, 2016Introduction to Artificial Intelligence Lecture 16: Neural Network Paradigms I 1 … let us move on to… Artificial Neural Networks.
Building a scalable neural processing subsystem Joy Bose Supervisors Steve Furber (Amulet Group) Jonathan Shapiro (AI Group) Amulet Group Meeting 30 January.
Summary Presentation of Cortical Computing with Memrisitve Nanodevices Authored by Greg S. Snider Hewlett-Packard Laboratories Published Winter 2008, SciDAC.
Classification of models
Optical RESERVOIR COMPUTING
Architecture & Organization 1
Architecture & Organization 1
Introduction to Neural Networks And Their Applications - Basics
Cerebral Cortex Intricate fabric of interconnected neural cells that covers the cerebral hemispheres. It is the body’s ultimate control and information.
III. Introduction to Neural Networks And Their Applications - Basics
ARTIFICIAL NEURAL networks.
Biological Based Networks
Machine Learning.
Summary Presentation of Cortical Computing with Memrisitve Nanodevices
Presentation transcript:

Hybrid Pipeline Structure for Self-Organizing Learning Array Yinyin Liu 1, Ding Mingwei 2, Janusz A. Starzyk 1, 1 School of Electrical Engineering & Computer Science Ohio University, USA 2 Ross University ISNN 2007: The 4th International Symposium on Neural Networks

2 Outline RC systems design of SOLAR Dimensionality reduction Input selection, weighting Pipeline structure Experimental results Conclusions Broca’s area Pars opercularis Motor cortex Somatosensory cortex Sensory associative cortex Primary Auditory cortex Wernicke’s area Visual associative cortex Visual cortex

3 “… Perhaps the last frontier of science – its ultimate challenge- is to understand the biological basis of consciousness and the mental process by which we perceive, act, learn and remember.. ” from Principles of Neural Science by E. R. Kandel et al. E. R. Kandel won Nobel Price in 2000 for his work on physiological basis of memory storage in neurons. “… The question of intelligence is the last great terrestrial frontier of science... ” from Jeff Hawkins On Intelligence. Jeff Hawkins founded the Redwood Neuroscience Institute devoted to brain research. He co-founded Palm Computing and Handspring Inc. Intelligence AI’s holy grail From Pattie Maes MIT Media Lab

4 How can we design intelligence? We need to know how We need means to implement it We need resources to build and sustain its operation

5 From Ray Kurzwail, The Singularity Summit at Stanford, May 13, 2006 Resources – Evolution of Electronics

6 By Gordon E. Moore

7

8 From Ray Kurzwail, The Singularity Summit at Stanford, May 13, 2006 Clock Speed (doubles every 2.7 years)

9 From Ray Kurzwail, The Singularity Summit at Stanford, May 13, 2006

10 Outline RC systems design of SOLAR Dimensionality reduction Input selection, weighting Pipeline structure Experimental results Conclusions Broca’s area Pars opercularis Motor cortex Somatosensory cortex Sensory associative cortex Primary Auditory cortex Wernicke’s area Visual associative cortex Visual cortex

11 Traditional ANN Hardware –Limited routing resource. –Quadratic relationship between the routing and the number of neuron makes classical ANNs wire dominated. input output information flow hidden Interconnect is 70% of chip area

12 Biological Neural Networks Biological Neural Networks Cell body From IFC’s webpage Dowling, 1998, p. 17

13 Sparse Structure neurons in human brain are sparsely connected On average, each neuron is connected to other neurons through about 10 4 synapses Sparse structure enables efficient computation and saves energy and cost

14 Why should we care? Source: SEMATECH

15 Percent of die area that must be occupied by memory to maintain SOC design productivity Design Productivity Gap  Low-Value Designs? Source = Japanese system-LSI industry

16 Outline RC systems design of SOLAR Dimensionality reduction Input selection, weighting Pipeline structure Experimental results Conclusions Broca’s area Pars opercularis Motor cortex Somatosensory cortex Sensory associative cortex Primary Auditory cortex Wernicke’s area Visual associative cortex Visual cortex

17 SOLAR System Design SOLAR Introduction Entropy based self- organization – data-driven – Local connection Dynamical reconfiguration Local and sparse i nterconnections Online inputs selection Feature neurons and merging neurons Pattern recognition, classification

18 Pipeline Overview node computing ability → “soft” connections Four modes 1. Idle 2. Read 3. Process 4. Write

19 Pipeline Signal Flow 1

20 Pipeline Signal Flow 2

21 Pipeline Signal Flow 3

22 Node Operations Implemented with Xilinx picoBlaze Runs at higher frequency

23 Outline RC systems design of SOLAR Dimensionality reduction Input selection, weighting Pipeline structure Experimental resultsExperimental results Conclusions Broca’s area Pars opercularis Motor cortex Somatosensory cortex Sensory associative cortex Primary Auditory cortex Wernicke’s area Visual associative cortex Visual cortex

24 Em(x) Simulation Results

25 Iris Data Processing 4x7 array processing Iris data Linear growth of HW cost

26 Chip Layout

27 XILINX VIRTEX XCV 1000 Hardware Development

28 Future Work - System SOLAR

29 Conclusions & Future work Sparse coding building in sparsely connected networks WTA scheme: local competition accomplish the global competition using primary and secondary layers –efficient hardware implementation OTA scheme: local competition produces neuronal activity reduction OTA – redundant coding: more reliable and robust WTA & OTA: learning memory for developing machine intelligence Future work: Introducing temporal sequence learning Building motor pathway on such learning memory Combining with goal-creation pathway to build intelligent machine