ECE 5984: Introduction to Machine Learning Dhruv Batra Virginia Tech Topics: –Regression Readings: Barber 17.1, 17.2.

Slides:

Advertisements

Similar presentations

ECE 5984: Introduction to Machine Learning Dhruv Batra Virginia Tech Topics: –(Finish) Regression –Model selection, Cross-validation –Error decomposition.

Advertisements

Parameter and Structure Learning Dhruv Batra, Recitation 10/02/2008.

INTRODUCTION TO MACHINE LEARNING Bayesian Estimation.

Middle Term Exam 03/01 (Thursday), take home, turn in at noon time of 03/02 (Friday)

ETHEM ALPAYDIN © The MIT Press, Lecture Slides for.

ECE 5984: Introduction to Machine Learning Dhruv Batra Virginia Tech Topics: –SVM –Multi-class SVMs –Neural Networks –Multi-layer Perceptron Readings:

INTRODUCTION TO Machine Learning ETHEM ALPAYDIN © The MIT Press, Lecture Slides for.

Bayesian wrap-up (probably). Administrivia My schedule has been chaos... Thank you for your understanding... Feedback on the student lectures? HW2 not.

Maximum Likelihood We have studied the OLS estimator. It only applies under certain assumptions In particular,  ~ N(0, 2 ) But what if the sampling distribution.

Machine Learning CUNY Graduate Center Lecture 3: Linear Regression.

Kernel Methods Part 2 Bing Han June 26, Local Likelihood Logistic Regression.

ECE 5984: Introduction to Machine Learning

Simple Bayesian Supervised Models Saskia Klein & Steffen Bollmann 1.

Bayes Classifier, Linear Regression 10701/15781 Recitation January 29, 2008 Parts of the slides are from previous years’ recitation and lecture notes,

Bayesian Wrap-Up (probably). Administrivia Office hours tomorrow on schedule Woo hoo! Office hours today deferred... [sigh] 4:30-5:15.

Crash Course on Machine Learning

ECE 5984: Introduction to Machine Learning

ECE 5984: Introduction to Machine Learning Dhruv Batra Virginia Tech Topics: –Classification: Naïve Bayes Readings: Barber

ECE 5984: Introduction to Machine Learning Dhruv Batra Virginia Tech Topics: –Probability Review –Statistical Estimation (MLE) Readings: Barber 8.1, 8.2.

CPSC 502, Lecture 15Slide 1 Introduction to Artificial Intelligence (AI) Computer Science cpsc502, Lecture 16 Nov, 3, 2011 Slide credit: C. Conati, S.

ECE 6504: Deep Learning for Perception Dhruv Batra Virginia Tech Topics: –Neural Networks –Backprop –Modular Design.

ECE 5984: Introduction to Machine Learning Dhruv Batra Virginia Tech Topics: –Statistical Estimation (MLE, MAP, Bayesian) Readings: Barber 8.6, 8.7.

CSE 446: Point Estimation Winter 2012 Dan Weld Slides adapted from Carlos Guestrin (& Luke Zettlemoyer)

Empirical Research Methods in Computer Science Lecture 7 November 30, 2005 Noah Smith.

CSE 446 Logistic Regression Winter 2012 Dan Weld Some slides from Carlos Guestrin, Luke Zettlemoyer.

ECE 5984: Introduction to Machine Learning Dhruv Batra Virginia Tech Topics: –Unsupervised Learning: Kmeans, GMM, EM Readings: Barber

ECE 5984: Introduction to Machine Learning Dhruv Batra Virginia Tech Topics: –(Finish) Nearest Neighbour Readings: Barber 14 (kNN)

ECE 5984: Introduction to Machine Learning Dhruv Batra Virginia Tech Topics: –Classification: Logistic Regression –NB & LR connections Readings: Barber.

ECE 5984: Introduction to Machine Learning Dhruv Batra Virginia Tech Topics: –Supervised Learning –General Setup, learning from data –Nearest Neighbour.

ECE 6504: Deep Learning for Perception Dhruv Batra Virginia Tech Topics: –Recurrent Neural Networks (RNNs) –BackProp Through Time (BPTT) –Vanishing / Exploding.

CORRECTIONS L2 regularization ||w|| 2 2, not ||w|| 2 Show second derivative is positive or negative on exams, or show convex – Latter is easier (e.g. x.

Machine Learning with Discriminative Methods Lecture 05 – Doing it 1 CS Spring 2015 Alex Berg.

INTRODUCTION TO MACHINE LEARNING 3RD EDITION ETHEM ALPAYDIN © The MIT Press, Lecture.

Over-fitting and Regularization Chapter 4 textbook Lectures 11 and 12 on amlbook.com.

ECE 5984: Introduction to Machine Learning Dhruv Batra Virginia Tech Topics: –Ensemble Methods: Bagging, Boosting Readings: Murphy 16.4; Hastie 16.

ECE 6504: Deep Learning for Perception Dhruv Batra Virginia Tech Topics: –LSTMs (intuition and variants) –[Abhishek:] Lua / Torch Tutorial.

Machine Learning CUNY Graduate Center Lecture 6: Linear Regression II.

ECE 5984: Introduction to Machine Learning Dhruv Batra Virginia Tech Topics: –Decision/Classification Trees Readings: Murphy ; Hastie 9.2.

CS 2750: Machine Learning Probability Review Prof. Adriana Kovashka University of Pittsburgh February 29, 2016.

ECE 6504: Deep Learning for Perception Dhruv Batra Virginia Tech Topics: –Deconvolution.

ECE 5984: Introduction to Machine Learning Dhruv Batra Virginia Tech Topics: –(Finish) Expectation Maximization –Principal Component Analysis (PCA) Readings:

ETHEM ALPAYDIN © The MIT Press, Lecture Slides for.

ECE 5984: Introduction to Machine Learning Dhruv Batra Virginia Tech Topics: –(Finish) Model selection –Error decomposition –Bias-Variance Tradeoff –Classification:

Matt Gormley Lecture 11 October 5, 2016

ECE 5424: Introduction to Machine Learning

ECE 5424: Introduction to Machine Learning

Probability Theory and Parameter Estimation I

ECE 5424: Introduction to Machine Learning

ECE 5424: Introduction to Machine Learning

ECE 5424: Introduction to Machine Learning

ECE 5424: Introduction to Machine Learning

Ch3: Model Building through Regression

CS 2750: Machine Learning Probability Review Density Estimation

ECE 5424: Introduction to Machine Learning

ECE 5424: Introduction to Machine Learning

ECE 5424: Introduction to Machine Learning

ECE 5424: Introduction to Machine Learning

ECE 5424: Introduction to Machine Learning

ECE 5424: Introduction to Machine Learning

Special Topics In Scientific Computing

ECE 5424: Introduction to Machine Learning

Probabilistic Models for Linear Regression

Distributions and Concepts in Probability Theory

ECE 5424: Introduction to Machine Learning

ECE 5424: Introduction to Machine Learning

OVERVIEW OF BAYESIAN INFERENCE: PART 1

Parametric Methods Berlin Chen, 2005 References:

Dhruv Batra Georgia Tech

Presentation transcript:

ECE 5984: Introduction to Machine Learning Dhruv Batra Virginia Tech Topics: –Regression Readings: Barber 17.1, 17.2

Administrativia HW1 –Due on Sun 02/15, 11:55pm – Project Proposal –Due: Tue 02/24, 11:55 pm –<=2pages, NIPS format HW2 –Out today –Due on Friday 03/06, 11:55pm –Please please please please please start early –Implement linear regression, Naïve Bayes, Logistic Regression (C) Dhruv Batra2

Recap of last time (C) Dhruv Batra3

Learning a Gaussian Collect a bunch of data –Hopefully, i.i.d. samples –e.g., exam scores Learn parameters –Mean –Variance (C) Dhruv Batra4

MLE for Gaussian Prob. of i.i.d. samples D={x 1,…,x N }: Log-likelihood of data: Slide Credit: Carlos Guestrin(C) Dhruv Batra5

Your second learning algorithm: MLE for mean of a Gaussian What’s MLE for mean? Slide Credit: Carlos Guestrin(C) Dhruv Batra6

Learning Gaussian parameters MLE: (C) Dhruv Batra7

Conjugate priors –Mean: Gaussian prior –Variance: Inverse Gamma or Wishart Distribution Prior for mean: Slide Credit: Carlos Guestrin(C) Dhruv Batra8 Bayesian learning of Gaussian parameters

MAP for mean of Gaussian Slide Credit: Carlos Guestrin(C) Dhruv Batra9

Plan for Today Regression –Linear Regression –Connections with Gaussians (C) Dhruv Batra10

New Topic: Regression (C) Dhruv Batra11

1-NN for Regression Often bumpy (overfits) (C) Dhruv Batra12Figure Credit: Andrew Moore

(C) Dhruv Batra13Slide Credit: Greg Shakhnarovich

(C) Dhruv Batra14Slide Credit: Greg Shakhnarovich

(C) Dhruv Batra15Slide Credit: Greg Shakhnarovich

Linear Regression Demo – eastSquaresDemo.htmlhttp://hspm.sph.sc.edu/courses/J716/demos/LeastSquares/L eastSquaresDemo.html (C) Dhruv Batra16

(C) Dhruv Batra17Slide Credit: Greg Shakhnarovich

(C) Dhruv Batra18Slide Credit: Greg Shakhnarovich

(C) Dhruv Batra19Slide Credit: Greg Shakhnarovich

(C) Dhruv Batra20Slide Credit: Greg Shakhnarovich

(C) Dhruv Batra21Slide Credit: Greg Shakhnarovich

(C) Dhruv Batra22Slide Credit: Greg Shakhnarovich

(C) Dhruv Batra23Slide Credit: Greg Shakhnarovich

Why sum squared error??? Gaussians, Watson, Gaussians… But, why? 24(C) Dhruv Batra

25Slide Credit: Greg Shakhnarovich

MLE Under Gaussian Model On board (C) Dhruv Batra26

Is OLS Robust? Demo – Bad things happen when the data does not come from your model! How do we fix this? (C) Dhruv Batra27

Robust Linear Regression y ~ Lap(w’x, b) On board (C) Dhruv Batra28