1. Setup for linear regression model

2. Analyzing “EPE” (fixed X) 0 0 (independence) =  2 (irreducible) bias 2 variance (=  2 p/n)

3. Variance y new y  Graphical picture of linear model Data variance F Irreducible error Bias

4. Regularization in linear regression Questions: What happens to our bias? What happens to our variance? What happens to our calculations (still orthogonal projections?)

5. Data variance Graphical picture of linear model + regularization  y y new F Irreducible error Bias F Model variance FcFc Variance