1. CSC 4510 – Machine Learning Dr. Mary-Angela Papalaskari Department of Computing Sciences Villanova University Course website: www.csc.villanova.edu/~map/4510/ 4: Regression 1CSC 4510 - M.A. Papalaskari - Villanova University T he slides in this presentation are adapted from: The Stanford online ML course http://www.ml-class.org/http://www.ml-class.org/

2. Housing Prices (Portland, OR) Price (in 1000s of dollars) Size (feet 2 ) data file

3. Housing Prices (Portland, OR) Price (in 1000s of dollars) Size (feet 2 ) Supervised Learning Given the “right answer” for each example in the data. Regression Problem Predict real-valued output

4. Notation: m = Number of training examples x’s = “input” variable / features y’s = “output” variable / “target” variable Size in feet 2 (x) Price ($) in 1000's (y) 2104460 1416232 1534315 852178 …… Training set of housing prices (Portland, OR)

5. Training Set Learning Algorithm h Size of house Estimate price

6. Training Set Learning Algorithm h Size of house Estimate price Linear Hypothesis: Univariate linear regression)

7. How to choose ‘s ? Training Set Hypothesis: ‘s: Parameters Size in feet 2 (x) Price ($) in 1000's (y) 2104460 1416232 1534315 852178 ……

10. Idea: Choose θ 0,θ 1 so that h θ (x) is close to y for our training examples What are good measures of being “close”? CSC 4510 - M.A. Papalaskari - Villanova University10

11. Hypothesis: Parameters: Cost Function: Goal:

13. (for fixed, this is a function of x)(function of the parameters )

14. (for fixed, this is a function of x)(function of the parameters )

15. (for fixed, this is a function of x)(function of the parameters )

16. (for fixed, this is a function of x)(function of the parameters )

17. Have some function Want Outline: Start with some Keep changing to reduce until we hopefully end up at a minimum

18.   J(     )

19.  

20. Next time: Gradient descent algorithm for linear univariate regression update and simultaneously Exercise: Let’s use Excel to find h(x) for the data file housing prices exampledata file (if you need a spreadsheet refresher try this: http://www.ncsu.edu/labwrite/res/gt/gt-reg-home.html#cal)http://www.ncsu.edu/labwrite/res/gt/gt-reg-home.html#cal