Setup for linear regression model

Analyzing “EPE” (fixed X) 0 (independence) =2 (irreducible) bias2 variance (= 2p/n)

Graphical picture of linear model Data variance y Variance Bias  Irreducible error ynew

Regularization in linear regression Questions: What happens to our bias? What happens to our variance? What happens to our calculations (still orthogonal projections?)

Graphical picture of linear model + regularization Data variance Model variance Fc y Variance Bias  Irreducible error ynew