1. SOLAR @ ICCINC'2003 1 Janusz Starzyk, Yongtao Guo and Zhineng Zhu Ohio University, Athens, OH 45701, U.S.A. 6 th International Conference on Computational Intelligence and Neural Computing Cary, NC, September 30 th, 2003

2. SOLAR @ ICCINC'2003 2 OUTLINE  Neural Networks  Traditional Hardware Implementation  Principle of Self-Organizing Learning  Advantages & Simulation Algorithm  Hardware Architecture  Hardware/software Codesign  Routing and Interface  PCB SOLAR  Future Work  Conclusion

3. SOLAR @ ICCINC'2003 3 Traditional ANN Hardware –Limited routing resource. –Quadratic relationship between the routing and the number of neuron makes classical ANNs wire dominated. input output information flow hidden Interconnect is 70% of chip area

4. SOLAR @ ICCINC'2003 4 Biological Neural Networks Biological Neural Networks Cell body From IFC’s webpage Dowling, 1998, p. 17

5. SOLAR @ ICCINC'2003 5 What is SOLAR?What is SOLAR? New Biologically Inspired Learning Network Organization Basic Fabric: A fixed lattice of distributed, parallel processing units (neurons) Self-organization: N eurons chose inputs adaptively from routing channels. N eurons chose inputs adaptively from routing channels. Neurons are adaptively self re-configured. Neurons are adaptively self re-configured. Neurons send output signals to the routing channels. Neurons send output signals to the routing channels. Number of neurons results automatically from problem complexity. Number of neurons results automatically from problem complexity. Self Organizing Learning Array SOLAR

6. SOLAR @ ICCINC'2003 6 Self Organizing Learning Array SOLAR-Organization  Neurons organized in a cell array  Sparse randomized connections  Local self-organization  Data driven  Entropy based learning  Regular structure  Suitable for large scale circuit implementation

7. SOLAR @ ICCINC'2003 7 Neuron’s Simulation Structure Neuron Inputs Neuron Inputs –System clock –Data input –Control input TCI –Information deficiency ID Other Neurons This neuron System clock Nearest neighbor neuron Remote neurons TCITCI IDID Neuron Outputs -Data output -Control output -Information deficiency

8. SOLAR @ ICCINC'2003 8 Self-Organizing Process

9. SOLAR @ ICCINC'2003 9 Self-organizing Principle Information index Neuron self-organizes by maximizing the information index

10. SOLAR @ ICCINC'2003 10 Self-organizing Principle Output information deficiency. Information deficiency (helps to organize SOLAR learning) The learning array grows by adding more neurons until input information deficiency of a subsequent neuron falls below threshold

11. Self-organizing Process Matlab Simulation Initial interconnection Learning process

12. SOLAR @ ICCINC'2003 12 Software Simulation Training Data SOLAR & other Algorithms Credit card approval data (ftp:cs.uci.edu) SOLAR & other Classifiers (Simulation) MethodMiss Detection Probability MethodMiss Detection Probability CAL5.131Naivebay.151 DIPOL92.141CASTLE.148 Logdisc.141ALLOC80.201 SMART.158CART.145 C4.5.155NewID.181 IndCART.152CN2.204 Bprop.154LVQ.197 RBF.145Quadisc.207 Baytree.171Default.440 ITule.137k-NN.181 AC2.181SOLAR.135

13. SOLAR @ ICCINC'2003 13 Structure of a single neuron RPU: reconfigurable processing unit CU: control unit DPE: dynamic probability estimator EBE: entropy based evaluator DSRU: dynamic self-reconfiguration memory. NI/NO: Data input/output CI/CO: Control input/output

14. SOLAR @ ICCINC'2003 14 Routing Structure –CSU:configurable switching unit –BRU: bidirectional routing unit

15. SOLAR @ ICCINC'2003 15 Configurable Switching Unit (CSU) CSU is used to realize flexible connections among neurons –Butterfly structure –CSU can take any number of inputs Even number of inputs Odd number of inputs

16. SOLAR @ ICCINC'2003 16 Configurable Switching Unit(cont’d) Random connections of neurons with branching ratio of 50% for 3*6 and 6*15 neurons array Routing resources used 62.7%Routing resources used 85.3%

17. SOLAR @ ICCINC'2003 17 Branching Ratio of 10%Branching Ratio of 90% Random connections of 4*7 neurons array with branching ratio of 10% and 90% Configurable Switching Unit(cont’d)

18. SOLAR @ ICCINC'2003 18 HW/SW Codesign Partition of System Co-simulation  Neuron’s architecture  System initialization, organization and management  Interface JTAG Programming Software run in PC PCI Bus Hardware Board Virtex XCV800FPGA dynamic configuration

19. SOLAR @ ICCINC'2003 19 Software Model In Behavioural VHDL Hardware Model In Structural VHDL SW/HW Co-simulation A software process –Written in behavioral VHDL A hardware process –Written in RTL VHDL which is synthesizable HW/SW communication –FSM and FIFOs

20. SOLAR @ ICCINC'2003 20 Hardware Architecture

21. SOLAR @ ICCINC'2003 21 Software Architecture System Design Data I/O API PCI FUNC Kernel Driver Ctrl I/O API Sys Func Hardware Access Function Data I/O matDIME_DMARead.dll matDIME_DMAWrite.dll matviDIME_ReadRegister.dll matviDIME_WriteRegister.dll … Ctrl I/O matCloseDIMEBoard.dll matConfigDIMEBoard.dll matOpenDIMEBoard.dll … PCI BUS

22. SOLAR @ ICCINC'2003 22 PCB Design Single SOLAR PCB contains 2x2 VIRTEX XCV1000 chips

23. SOLAR @ ICCINC'2003 23 SOLAR PCB Design Boards Interface BoardSOLAR Board

24. SOLAR @ ICCINC'2003 24 Neurons Prototyping Problem: Neurons need to be carefully placed - otherwise some resources are lost. Neurons memory needs to be optimized for best resource utilization.

25. SOLAR @ ICCINC'2003 25 Future Work - System SOLAR

26. SOLAR @ ICCINC'2003 26 SOLAR is different from traditional neural networks …  Expandable modular architecture  Dynamically reconfigurable hardware structure  Interconnection number grows linearly with the number of neurons  Data-driven self-organizing learning hardware  Learning and organization is based on local information

27. SOLAR @ ICCINC'2003 27 Why to focus on networks of neurons?  Increases computational speed  Improves fault tolerance  Constraints us to use distributed solutions  Brain does it  www.ent.ohiou.edu/~starzyk

28. SOLAR @ ICCINC'2003 28 Can we set milestones in developing intelligent networks of neurons? How to represent a distributed cognition? How to model machine will to learn and act? How to introduce association between patterns? How a machine shell implement temporal learning? How machine shell block repetitive information from being processed over and over again? How machine shell evaluate its state with respect to set objectives and plan its actions? How to implement elements of reinforcement learning in distributed networks?

29. SOLAR @ ICCINC'2003 29 Questions