On L1q Regularized Regression Authors: Han Liu and Jian Zhang Presented by Jun Liu

Problem (1)

Problem (2) The number of groups is much larger than the number of samples

Outline Proposition 2.1, 2.2 (Subgradient, linearly dependent) Definition (Properties to be established) Theorem 3.1 (Variable Selection Consistency) Lemma 4.1 (Technical lemma) Assumption 1, Theorem 4.3 (Consistency, linear model) Assumption 2. Theorem 4.5 (Inequality, misspecified model) Assumption 4, Theorem 5.1 (Risk consistency)

We want to find the such that the q’-norm of is either equal to a constant value (for nonzero groups) or bounded (for zero groups)

Scale invariant, Sign Preserving Scale invariant, Sign Preserving Scale invariant, Sign Preserving

Hint This result is similar to the Lasso. The key is that, so that any m>n columns of X are linearly dependent.

g j, j=2, …, s is not changed

Outline Proposition 2.1, 2.2 (Subgradient, linear dependent) Definition (Properties to be established) Theorem 3.1 (Variable Selection Consistency) Lemma 4.1 (Technical lemma) Assumption 1, Theorem 4.3 (Consistency, linear model) Assumption 2. Theorem 4.5 (Inequality, misspecified model) Assumption 4, Theorem 5.1 (Risk consistency)

Outline Proposition 2.1, 2.2 (Subgradient, linear dependent) Definition (Properties to be established) Theorem 3.1 (Variable Selection Consistency) Lemma 4.1 (Technical lemma) Assumption 1, Theorem 4.3 (Consistency, linear model) Assumption 2. Theorem 4.5 (Inequality, misspecified model) Assumption 4, Theorem 5.1 (Risk consistency)

Key Points in the Proof Objective Two parts Tools Proof by construction Solution is not unique

Part 1

Part 2

Outline Proposition 2.1, 2.2 (Subgradient, linear dependent) Definition (Properties to be established) Theorem 3.1 (Variable Selection Consistency) Lemma 4.1 (Technical lemma) Assumption 1, Theorem 4.3 (Consistency, linear model) Assumption 2. Theorem 4.5 (Inequality, misspecified model) Assumption 4, Theorem 5.1 (Risk consistency)

Outline Proposition 2.1, 2.2 (Subgradient, linear dependent) Definition (Properties to be established) Theorem 3.1 (Variable Selection Consistency) Lemma 4.1 (Technical lemma) Assumption 1, Theorem 4.3 (Consistency, linear model) Assumption 2. Theorem 4.5 (Inequality, misspecified model) Assumption 4, Theorem 5.1 (Risk consistency)