1. On L1q Regularized Regression Authors: Han Liu and Jian Zhang Presented by Jun Liu

2. Problem (1)

3. Problem (2) The number of groups is much larger than the number of samples

4. Outline Proposition 2.1, 2.2 (Subgradient, linearly dependent) Definition 2.4-2.7 (Properties to be established) Theorem 3.1 (Variable Selection Consistency) Lemma 4.1 (Technical lemma) Assumption 1, Theorem 4.3 (Consistency, linear model) Assumption 2. Theorem 4.5 (Inequality, misspecified model) Assumption 4, Theorem 5.1 (Risk consistency)

5. We want to find the such that the q’-norm of is either equal to a constant value (for nonzero groups) or bounded (for zero groups)

6. Scale invariant, Sign Preserving Scale invariant, Sign Preserving Scale invariant, Sign Preserving

7. Hint This result is similar to the Lasso. The key is that, so that any m>n columns of X are linearly dependent.

8. g j, j=2, …, s is not changed

9. Outline Proposition 2.1, 2.2 (Subgradient, linear dependent) Definition 2.4-2.7 (Properties to be established) Theorem 3.1 (Variable Selection Consistency) Lemma 4.1 (Technical lemma) Assumption 1, Theorem 4.3 (Consistency, linear model) Assumption 2. Theorem 4.5 (Inequality, misspecified model) Assumption 4, Theorem 5.1 (Risk consistency)

13. Outline Proposition 2.1, 2.2 (Subgradient, linear dependent) Definition 2.4-2.7 (Properties to be established) Theorem 3.1 (Variable Selection Consistency) Lemma 4.1 (Technical lemma) Assumption 1, Theorem 4.3 (Consistency, linear model) Assumption 2. Theorem 4.5 (Inequality, misspecified model) Assumption 4, Theorem 5.1 (Risk consistency)

16. Key Points in the Proof Objective Two parts Tools Proof by construction Solution is not unique

18. Part 1

21. Part 2

24. Outline Proposition 2.1, 2.2 (Subgradient, linear dependent) Definition 2.4-2.7 (Properties to be established) Theorem 3.1 (Variable Selection Consistency) Lemma 4.1 (Technical lemma) Assumption 1, Theorem 4.3 (Consistency, linear model) Assumption 2. Theorem 4.5 (Inequality, misspecified model) Assumption 4, Theorem 5.1 (Risk consistency)

29. Outline Proposition 2.1, 2.2 (Subgradient, linear dependent) Definition 2.4-2.7 (Properties to be established) Theorem 3.1 (Variable Selection Consistency) Lemma 4.1 (Technical lemma) Assumption 1, Theorem 4.3 (Consistency, linear model) Assumption 2. Theorem 4.5 (Inequality, misspecified model) Assumption 4, Theorem 5.1 (Risk consistency)