1. CBP 2009-10Comp 3104 The Nature of Computing 1 Nature Inspired Computing Artificial Neural Nets - Symbiosis between computer and cognitive sciences

2. CBP 2009-10Comp 3104 The Nature of Computing 2 Cajal -1- CajalGolgi Cajal + Golgi indentification of independent neurons by staining, microscopy and looking. (Nobel Prize 1906)

3. CBP 2009-10Comp 3104 The Nature of Computing 3 Cajal -2-

4. CBP 2009-10Comp 3104 The Nature of Computing 4 Rat Neurons

5. CBP 2009-10Comp 3104 The Nature of Computing 5 Neuron Grows on Electrodes Copyright © 2000 Yoonkey Nam - Department of Electrical & Computer Engineering; Imaging Technology Group; Beckman Institute; and the University of Illinois

6. CBP 2009-10Comp 3104 The Nature of Computing 6 Investigation of Single Neurons Microelectrode recording of Biological Neuron activation using tungsten electrode Hubel and Weisel. Nobel Prize 1958 Photomicrograph: Height = 1mm.

7. CBP 2009-10Comp 3104 The Nature of Computing 7 Biological Neurons dendrites axon synapse Signal flowBig Neurological principle #1 Neurons work using electricity, not blood or other special goo Signal shape

8. CBP 2009-10Comp 3104 The Nature of Computing 8 Single Neuron In 2 In 1 In 3 In 4 In 1 + In 2 + In 3 + In 4 “input” “activation” “input” A A B B “threshold” Big Neurological principle #2 “Integrate and Fire” Inputs summed. If above threshold output fires.

9. CBP 2009-10Comp 3104 The Nature of Computing 9 Learning in Neural Nets Before Learning After Learning Big Neurological principle #3 “Hebbian Learning” Synapse strength increases if both cells A and B are firing A A B B

10. CBP 2009-10Comp 3104 The Nature of Computing 10 Brains Minds and Computers BrainsComputers Work using Electricity Have inputs and outputs Can learn by experience Can be taught Work using Electricity Have inputs and outputs Can be programmed So do we understand brains? Yep. Do we therefore understand Minds?Nope.

11. CBP 2009-10Comp 3104 The Nature of Computing 11 Artificial Neurons In 1 + In 2 + In 3 + In 4 In 1 In 2 In 3 “output” “input” A A B B “threshold” inputs output

12. CBP 2009-10Comp 3104 The Nature of Computing 12 Learning Logical Gates 0.5 Threshold = 1 1 1 1 1.5 Ouput neuron fires only when sum is greater than the threshold AND - gate OR - gate ABO 000 010 100 111 ABO 000 011 101 111

13. CBP 2009-10Comp 3104 The Nature of Computing 13 Training an Artificial Neural Net eyesmotors right left 1. We want to train the robot to move towards the light 2. So when the right eye gets light, the left motor neuron must fire and vice versa 3. We must therefore strengthen the cross - connections and kill the direct connections

14. CBP 2009-10Comp 3104 The Nature of Computing 14 Back Propagation of Errors eyesmotors right left 10.5 eyesmotors right left 1 0.5 Let’s say desired motor drive is 1.0 (full forwards). Here the right motor drive should be 0 but it is 0.5. So the error is Desired - actual = 0.0 - 0.5 = -0.5 So we decrease the connection by 0.5 Here the right motor drive should be 1.0 but it is 0.5. So the error is Desired - actual = 1.0 - 0.5 = +0.5 So we increase the connection by 0.5

15. CBP 2009-10Comp 3104 The Nature of Computing 15 Neural Net Solver

16. CBP 2009-10Comp 3104 The Nature of Computing 16 Medical Application Flu Neural Net cough headache

17. CBP 2009-10Comp 3104 The Nature of Computing 17 Medical Diagnosis Cough Headache Meningitis Flu Pneuomonia Not ill 1 Cough Headache 1 Flu 1

18. CBP 2009-10Comp 3104 The Nature of Computing 18 “Classical” Medical Diagnosis If ( (symptom ! = cough) && (symptom != headache) ) illness = no illness; else if ( (symptom ! = cough) && (symptom == headache) ) illness = meningitis; else if ( (symptom == cough) && (symptom != headache) ) illness = pneumonia; else if ( (symptom == cough) && (symptom == headache) ) illness = flu; Rule-based Learning “ if … then …. else … “

19. CBP 2009-10Comp 3104 The Nature of Computing 19 ECG Interpretation

20. CBP 2009-10Comp 3104 The Nature of Computing 20 NNets vs Expert Systems ModelingExamplesExplanation EffortNeededProvided Rule-based Exp. Syst.highlowhigh Bayesian Netshighlowmoderate Classification Treeslowhigh“high” Neural Netslowhighlow Regression Modelshighmoderatemoderate