1. Rule Selection as Submodular Function
Wentao Ding

2. Rule Selection Problem
A combination of two weighted coverage functions.
maximize 𝐻 𝑃𝑜𝑠 𝑌 ,𝑁𝑒𝑔 𝑌 s.t. 𝑗: 𝑡 𝑗 ∈𝑅 𝑖 𝑥 𝑖 > 𝑦 𝑗 𝑦 𝑗 ∈ 0,1 𝑥 𝑖 ∈ 0,1 R1 R2 R3 R4
: Positive
: Negative

3. Submodular Function Submodular: Maximize a submodular function
Def 1: ∀𝐴⊆𝐵. 𝑓 𝐴∪ 𝑥 −𝑓 𝐴 ≥𝑓 𝐵∪ 𝑥 −𝑓 𝐵
Def 2: ∀𝐴⊆𝐵. 𝑓 𝐴 +𝑓 𝐵 ≥𝑓 𝐴∪𝐵 +𝑓 𝐴∩𝐵
Coverage function is monotone submodular function.
Maximize a submodular function
usually NP-hard.
½-approx if symmetric, 𝑒−1 𝑒 -approx if monotone.
Combination of two submodular functions
Close under non-negative linear combinations.
Difference of Submodular function: Inapproximable.
Ratio of Submodular function: Depends

4. Rule Section Problem
Let #𝑝𝑜𝑠𝑡𝑖𝑣𝑒 #𝑎𝑙𝑙 =𝐶, 𝑁𝑒𝑔 𝑌 #𝑎𝑙𝑙 =𝑓 𝑌 , 𝑃𝑜𝑠 𝑌 #𝑎𝑙𝑙 =𝑔 𝑌 . TP=𝑔 𝑌 FP=𝑓 𝑌 TN=1−𝐶−𝑓 𝑌 FN=𝐶−𝑔 𝑌 𝑃𝑟𝑒𝑐𝑖𝑠𝑖𝑜𝑛= TP TP+FP = 𝑔 𝑌 𝑓 𝑌 +𝑔 𝑌 𝑅𝑒𝑐𝑎𝑙𝑙= TP TP+FN = 𝑔 𝑌 𝐶 𝑨𝒄𝒄𝒖𝒓𝒂𝒄𝒚= TP+TN 1 =1−𝐶+𝑔 𝑌 −𝑓 𝑌 𝑭 𝟏 = 2TP 2TP+FP+FN = 2𝑔 𝑌 𝐶+𝑓 𝑌 +𝑔 𝑌 ≈ 𝑔 𝑌 𝑓 𝑌

5. Ratio of submodular function
min 𝑁 𝑌 𝑃 𝑌 ∃𝐶.𝑃 𝑌 ≥𝐶 min 𝑁 𝑌 𝐶 max 𝑃 𝑌 𝑁 𝑌 ∃𝐵.𝑁 𝑌 ≤𝐵 max 𝑃 𝑌 𝐵
A Submodular Cover (SCSC) / Submodular Knapsack (SCSK)
max 𝑓 𝑋 𝑋⊆𝑉∧𝑔 𝑋 ≥𝐶 𝑓,𝑔:submodular
max 𝑔 𝑋 𝑋⊆𝑉∧𝑓 𝑋 ≤𝐵 𝑓,𝑔:submodular
Can be approximated by ratio based greedy method

6. Curvature of submodular function
𝒦 𝑓 =1− min 𝑣∈𝑉 𝑓 𝑣 𝑉∖𝑣 𝑓 𝑣 ∈ 0,1
Approximate ratio with simple greedy algorithm
𝑓 𝑌 𝐺 𝑔 𝑌 𝐺 ≤ 1 1− exp 𝒦 𝑓 −1 ⋅ 𝑓 𝑌 ∗ 𝑔 𝑌 ∗
(if 𝒦 𝑃 =0, there exists an 𝜖-bounded scheme) R1 R2 R3 R4
𝒦 𝑔 = 1 2
𝒦 𝑓 =1
a (1+)-approximation for RS minimization in O(log(1/)) calls to the subroutine

7. Optimizing RS Ellipsoid Approximation：O 𝑛 log 𝑛 -approx.
Related Learning Problem
Submodular optimization
Feature selection
Data subset selection (Document Summarization) ……

8. Reference
George L. Nemhauser, Laurence A. Wolsey, Marshall L. Fisher: An analysis of approximations for maximizing submodular set functions - I. Math. Program. 14(1): (1978) Michele Conforti, Gérard Cornuéjols: Submodular set functions, matroids and the greedy algorithm: Tight worst-case bounds and some generalizations of the Rado-Edmonds theorem. Discrete Applied Mathematics 7(3): (1984) Mukund Narasimhan, Jeff A. Bilmes: A Submodular-supermodular Procedure with Applications to Discriminative Structure Learning. UAI 2005: Rishabh K. Iyer, Jeff A. Bilmes: Submodular Optimization with Submodular Cover and Submodular Knapsack Constraints. NIPS 2013: Rishabh K. Iyer, Jeff A. Bilmes: Algorithms for Approximate Minimization of the Difference Between Submodular Functions, with Applications. UAI 2012: Wenruo Bai, Rishabh K. Iyer, Kai Wei, Jeff A. Bilmes: Algorithms for Optimizing the Ratio of Submodular Functions. ICML 2016:

9. Thanks for listening
Q & A