1. Perceptron Learning Rule

2. Learning Rules Network is provided with a set of examples
• Supervised Learning
Network is provided with a set of examples
of proper network behavior (inputs/targets)
Perceptron Learning falls here.
• Reinforcement Learning
Network is only provided with a grade, or score,
which indicates network performance
Not very common, Used in control system applications
• Unsupervised Learning – No target available
Only network inputs are available to the learning
algorithm. Network learns to categorize (cluster)
the inputs.
Vector Quantization

3. Perceptron ArchitectureW w 1 , 2 R S = W w T 1 2 S = w i 1 , 2 R =

4. Single-Neuron Perceptron

5. Decision Boundary
• All points on the decision boundary have the same inner product with the weight vector.
• Therefore they have the same projection onto the weight vector, and they must lie on a line orthogonal to the weight vector

6. Example - OR

7. OR Solution
Weight vector should be orthogonal to the decision boundary.
Pick a point on the decision boundary to find the bias.

8. Multiple-Neuron Perceptron
Each neuron will have its own decision boundary.
A single neuron can classify input vectors into two categories.
A multi-neuron perceptron can classify input vectors into 2S categories.

9. Learning Rule Test Problem

10. Starting Point Random initial weight: Present p1 to the network:
Incorrect Classification.

11. Tentative Learning Rule
• Set 1w to p1
– Not stable
• Add p1 to 1w
Tentative Rule:

12. Second Input Vector
(Incorrect Classification)
Modification to Rule:

13. Patterns are now correctly classified.
Third Input Vector
(Incorrect Classification)
Patterns are now correctly classified.

14. Unified Learning Rule
A bias is a
weight with
an input of 1.

15. Multiple-Neuron Perceptrons
To update the ith row of the weight matrix:
Matrix form:

16. Apple/Banana Example Training Set Initial Weights First Iteration e t1 a – =

17. Second Iteration

18. Check

19. Perceptron Rule Capability
The perceptron rule will always converge to weights which accomplish the desired classification, assuming that such weights exist.

20. Perceptron Limitations
Linear Decision Boundary
Linearly Inseparable Problems