Download presentation
Presentation is loading. Please wait.
1
Linear Learning Machines Simplest case: the decision function is a hyperplane in input space. The Perceptron Algorithm: Rosenblatt, 1956 An on-line and mistake-driven procedure Update the weight vector and bias when there is a misclassified point Converge when problem is linearly separable
2
Basic Notations Input space: Output space: Hypothesis: Real-valued function: Training set:
3
Basic Notations Inner product: Norm: 1-norm: 2-norm: -norm:
4
Definition of Margin The (functional) margin of a training point with respect to a hyperplane to be the quantity: whereis called the weight vector and is called the bias Note: implies classify the point correctly
5
Definition of Margin The minimum of the margin distribution is the The geometric margin (functional) margin of The (functional) margin distribution of a hyperplane with respect to a training set is the distribution of the margins of the examples in w.r.t. a training set functional margin
6
Definition of Margin The margin of a training set is the maximum Note: For a linear separable training set, its margin will be positive geometric margin over all hyperplanes The hyperplane realizing this maximum is known as maximal margin hyperplanes
8
The Perceptron Algorithm Rosenblatt, 1956 Given a linearly separable training set and learning rate and the initial weight vector, bias: and let
9
The Perceptron Algorithm Repeat: until no mistakes made within the for loop return:. What is ?
10
The Perceptron Algorithm ( STOP in Finite Steps ) Theorem 2.3 (Novikoff) Let be a non-trivial training set, and let Suppose that there exists a vector and. Then the number of mistakes made by the on-line perceptron algorithm on is at most
Similar presentations
© 2025 SlidePlayer.com. Inc.
All rights reserved.