Setup for linear regression model

Analyzing “EPE” (fixed X) 0 0 (independence) =  2 (irreducible) bias 2 variance (=  2 p/n)

Variance y new y  Graphical picture of linear model Data variance F Irreducible error Bias

Regularization in linear regression Questions: What happens to our bias? What happens to our variance? What happens to our calculations (still orthogonal projections?)

Data variance Graphical picture of linear model + regularization  y y new F Irreducible error Bias F Model variance FcFc Variance