1. Presented by Scott Lichtor An Introduction to Neural Networks

2. Motivation I found for Neural Networks Pavlov’s dog Simple->Complex Learning

3. Overview Basics of the Nervous System – Neurons – Synapses – Action Potentials Neural Networks – Abstract Neurons – More Complicated Neurons – Learning – Supervised – Unsupervised – Reinforcement Conclusion

4. Basics of the Nervous System The nervous system coordinates the actions of an animal Body parts send messages to the brain Brain sends messages to body parts The basic unit of the nervous system is the neuron

5. Neurons Receive messages at the dendrites Message is sent quickly down the axon using electrical impulses What happens when the signal reaches the end of the axon? Image taken from img460.imageshack.us

6. Synapses Chemical Synapses Slow Strong Can be transmitted over long distances Image taken from http://www.airlinesafety.com/editorials

7. Synapses Electrical Synapses Very fast Fade quickly Image taken from wikipedia.org

8. Action Potentials Action potentials are shocks to a particular neuron The shock travels along the affected neuron Then, the action potential is transmitted from the affected neuron to the neurons connected to it The shock is transmitted to its destination in the same fashion

9. Abstract Neurons So biological neurons can be used to send modified messages from place to place Can be used to accomplish very complex tasks using relatively simple parts Can neurons represent other things/be used for other objectives?

10. Abstract Neurons Neurons can represent neuron-like things Inputs -> Processes -> Outputs Image taken from http://3.bp.blogspot.com/

11. Abstract Neurons Can “train” the neurons Neurons fire (output 1) under certain patterns Don’t fire (output 0) under other patterns Firing rule: if an outcome doesn’t fit in either pattern, it fires if it has more in common with the first set, and doesn’t fire if it has more in common with the second set. If there’s a tie, the neuron may fire, or it may not

12. Abstract Neurons Example A neuron takes three inputs (X1, X2, X3) The neuron is trained to output 1 if the inputs are 111 or 101 Trained to output 0 if the inputs are 000 or 001 Before firing rule: After firing rule: X100001111 X200110011 X301010101 Out000/1 1 1 X100001111 X200110011 X301010101 Out0000/1 111

13. Abstract Neurons The abstract neuron model can be used for pattern recognition Example: determine whether a ‘T’ or ‘H’ is displayed Can we model more complicated processes with neurons?

14. More Complicated Neurons McCulloch and Pitts model Difference from previous model: inputs are weighted. Add weighted inputs together: if the sum is greater than a threshold, then the neuron fires Image taken from http://www.doc.ic.ac.uk/~nd/surprise_96/journal/vol4/cs11 /report.html

15. More Complicated Neurons Mathematically: neuron fires if X 1 W 1 + X 2 W 2 + X 3 W 3 +... > T

16. Examples AND GateXOR Gate Image taken from http://www.heatonresearch.com

17. More Complicated Neurons New model is very adaptable/powerful Input weights and threshold can be changed so the neuron responds differently/more accurately to a situation Pavlov’s dog Various algorithms adapt neurons and neural networks to situations Delta rule (feed-forward networks) Back-error projection (feedback networks)

18. Learning For the network to adapt, it must learn. There are three types of learning used with neural networks: Supervised learning Unsupervised learning Reinforcement learning

19. Supervised Learning In supervised learning, the system learns using test data given from an external teacher The test data tells the system what outputs result from certain inputs The system tries to match the response of the test data, i.e. minimize the error between the neural network outputs and the test data outputs given the same inputs Image taken from http://www.learnartificialneuralnetworks.com

20. Unsupervised Learning In unsupervised learning, the network is given no output data Instead, the network is given just input data The goal of the network, then, is to group the input data Example: mortgage requests The network is given credit ratings, size of mortgage, interest rate, etc. The network groups the data; probably into accept and deny

21. Reinforcement Learning Network performs actions on the input data The environment grades the network (good or bad) The network makes adjustments accordingly Middle ground between supervised and unsupervised learning

22. Conclusion The learning aspect of neural networks makes their applications astounding For computers, one has to know how to solve a particular problem Neural networks can solve problems that one doesn’t know how to solve

23. Conclusion Just some of the uses: sales forecasting, stock market prediction, customer research, modeling and diagnosing the cardiovascular system, “Instant Physician”, interpretation of multi-meaning Chinese words, facial recognition, etc. etc. etc. Something I found interesting: the interconnectedness of different subjects

24. Sources http://www.doc.ic.ac.uk/~nd/surprise_96/journal/vol4/cs11 /report.html http://www.doc.ic.ac.uk/~nd/surprise_96/journal/vol4/cs11 /report.html http://www.learnartificialneuralnetworks.com/ http://www.ryerson.ca/~dgrimsha/courses/cps721/unsupervis ed.html http://www.ryerson.ca/~dgrimsha/courses/cps721/unsupervis ed.html http://www.willamette.edu/~gorr/classes/cs449/intro.html http://www.statsoft.com/textbook/stneunet.html http://www.wikipedia.org