1. Lasso/LARS summary
Nasimeh Asgarian

2. Lasso Summary Least Absolute Shrinkage and Selection operator
Given a set of input measurements x1,x2, …,xp and outcome measurement y, the lasso fits a linear model:
ŷ = 0+1*x1+2*x2+…+ p*xp
By minimizing
((y-ŷ)2)
Subject to
 | j| <= s

3. Computation of the lasso solution
Start with all j = 0
Find the predictor xj most correlated with y and add it to the model
Take residuals r = y – ŷ
Continue, at each stage add the predictor most correlated with r, to the model
Until all predictors are in the model

4. Lars Summary Least Angel Regression
Lasso is a restricted version of Lars
By minimizing L(, ) = ||y -  * X||2 +  ||1
LARS: uses least square directions in the active set of variables.
Lasso: uses least square directions; if a variable crosses zero, it is removed from the active set.

5. Computation of the Lars solution:
Start with all j = 0
Find the predictor xj most correlated with y
Increase the coefficient j in the direction of the sign of its correlation with y
Take residuals r = y – ŷ

6. Computation of the lasso solution: Lars (Least Angel Regression)
Stop when some other predictor xk has as much correlation with r as xj has.
Increase (j,k) in their joint least square direction, until some other predictor xm has as much correlation with the residual r.
Continue until all predictors are in the model.

7. Lasso: choice of tuning parameters
At each step of LOO CV,
Do 10-fold CV, on training set (twice)
Find optimal values of  and number of iteration based on 10-fold CV result. i.e. see which  value and how many number of steps gives maximum correlation coefficient.
Choose this  and number of iteration to build the model for the test instance.