Based on slides by William Cohen, Andrej Karpathy, Piyush Rai Linear Classifiers Based on slides by William Cohen, Andrej Karpathy, Piyush Rai

Linear Classifiers Let’s simplify life by assuming: Every instance is a vector of real numbers, x=(x1,…,xn). (Notation: boldface x is a vector.) First we consider only two classes, y=(+1) and y=(-1) A linear classifier is vector w of the same dimension as x that is used to make this prediction:

Visually, x · w is the distance you get if you “project x onto w” In 3d: lineplane In 4d: planehyperplane … X2 . w The line perpendicular to w divides the vectors classified as positive from the vectors classified as negative. -W

w -W Wolfram MathWorld Mediaboost.com

where b=w0 is called bias Notice that the separating hyperplane goes through the origin…if we don’t want this we can preprocess our examples: or where b=w0 is called bias

Back to Image Classification

3072 numbers in total reshaped into a column vector x

Interactive Web Demo: http://vision.stanford.edu/teaching/cs231n/linear-classify-demo/

Perceptron learning B A Compute: yi = sign(wk . xi ) ^ instance xi B A If mistake: wk+1 = wk + yi xi yi ^ yi 1957: The perceptron algorithm by Frank Rosenblatt 1960: Perceptron Mark 1 Computer – hardware implementation 1969: Minksky & Papert book shows perceptrons limited to linearly separable data 1970’s: learning methods for two-layer neural networks

Question