S. Mandayam/ ANN/ECE Dept./Rowan University Smart Sensors / Spring 2004 Shreekanth Mandayam ECE Department Rowan University Artificial Neural Networks Lecture 1 March 1, 2004

S. Mandayam/ ANN/ECE Dept./Rowan UniversityPlan What is artificial intelligence? Course module objectives Historical development – the neuron model The artificial neural network paradigm What is knowledge? What is learning? The Perceptron The “Future”….?

S. Mandayam/ ANN/ECE Dept./Rowan University Artificial Intelligence Systems that think like humans Cognitive modeling Systems that think rationally Logic Systems that act like humans Natural language processing Knowledge representation Machine learning Systems that act rationally Decision theoretic agents

S. Mandayam/ ANN/ECE Dept./Rowan UniversityObjectives At the conclusion of this course module the student will be able to: Identify and describe engineering paradigms for knowledge and learning Identify, describe and design artificial neural network architectures for simple cognitive tasks

S. Mandayam/ ANN/ECE Dept./Rowan University Biological Origins

S. Mandayam/ ANN/ECE Dept./Rowan University Biological Origins

S. Mandayam/ ANN/ECE Dept./Rowan UniversityHistory/People 1940’sTuringGeneral problem solver, “Turing test” 1940’sShannonInformation theory 1943McCulloch and PittsMath of neural processes 1949HebbLearning model 1959RosenblattThe “Perceptron” 1960WidrowLMS training algorithm 1969Minsky and PapertPerceptron deficiency 1985RumelhartFeedforward MLP, backprop 1988Broomhead and LoweRadial basis function neural nets 1990’sVLSI implementations

S. Mandayam/ ANN/ECE Dept./Rowan University Neural Network Paradigm Stage 1: Network Training ArtificialNeuralNetwork Present Examples Indicate Desired Outputs Determine Synaptic Weights ArtificialNeuralNetwork New Data Predicted Outputs Stage 2: Network Testing “knowledge”

S. Mandayam/ ANN/ECE Dept./Rowan University ANN Model ArtificialNeuralNetwork x Input Vector y Output Vector f Complex Nonlinear Function f(x) = y “knowledge”

S. Mandayam/ ANN/ECE Dept./Rowan University Popular I/O Mappings ANN x y Single output y1y1 ANN x 1-out-of-c selector y2y2 ycyc y1y1 ANN x Coder y2y2 ycyc ANN x Associator y

S. Mandayam/ ANN/ECE Dept./Rowan University The Perceptron    (.) w k1 w k2 w km x1x1 x2x2 xmxm Inputs Synaptic weights Bias, b k Induced field, v k Output, y k ukuk Activation/ squashing function

S. Mandayam/ ANN/ECE Dept./Rowan University“Learning” [w] x y ANN Mathematical Model of the Learning Process [w] 0 x y(0) Intitialize: Iteration (0) [w] 1 x y(1) Iteration (1) [w] n x y(n) = d Iteration (n) desired o/p

S. Mandayam/ ANN/ECE Dept./Rowan University Learning Rules Error Correction Learning Delta Rule or Widrow-Hoff Rule Memory Based Learning Nearest Neighbor Rule Hebbian Learning Competitive Learning Boltzman Learning

S. Mandayam/ ANN/ECE Dept./Rowan University Error-Correction Learning   (.) w k1 (n) x 1 (n) x2x2 xmxm Inputs Synaptic weights Bias, b k Induced field, v k (n) Activation/ squashing function w k2 (n) w km (n)  Output, y k (n) Desired Output, d k (n) Error Signal e k (n) + -

S. Mandayam/ ANN/ECE Dept./Rowan University Learning Paradigms Environment (Data) Teacher (Expert)  ANN error desired actual + - Supervised Unsupervised

S. Mandayam/ ANN/ECE Dept./Rowan University Learning Paradigms Supervised Unsupervised Environment (Data) Delay ANN Delayed Reinforcement Learning Cost Function

S. Mandayam/ ANN/ECE Dept./Rowan University Learning Tasks Pattern Association Pattern Recognition Function Approximation Filtering Classification x1x1 x2x2 1 2 DB x1x1 x2x2 1 2

S. Mandayam/ ANN/ECE Dept./Rowan University Perceptron Training Widrow-Hoff Rule (LMS Algorithm) w(0) = 0 n = 0 y(n) = sgn [w T (n) x(n)] w(n+1) = w(n) +  [d(n) – y(n)]x(n) n = n+1 Matlab Demo

S. Mandayam/ ANN/ECE Dept./Rowan University The Age of Spiritual Machines When Computers Exceed Human Intelligence by Ray Kurzweil | Penguin paperback | |

S. Mandayam/ ANN/ECE Dept./Rowan UniversitySummary