5 Maximizing submodular functions Minimizing convex functions: Polynomial time solvable! Minimizing submodular functions: Polynomial time solvable! Maximizing convex functions: NP hard! Maximizing submodular functions: NP hard! But can get approximation guarantees

6 Example: Set cover Node predicts values of positions with some radius For A µ V: z(A) = “area covered by sensors placed at A” Formally: W finite set, collection of n subsets S i µ W For A µ V={1,…,n} define z(A) = |  i2 A S i | Want to cover floorplan with discs Place sensors in building Possible locations V

7 Set cover is submodular S1S1 S2S2 S1S1 S2S2 S3S3 S4S4 S’ A={S 1,S 2 } B = {S 1,S 2,S 3,S 4 } z(A [ {S’})-z(A) z(B [ {S’})-z(B) ¸

8 Example: Feature selection Given random variables Y, X 1, … X n Want to predict Y from subset X A = (X i 1,…,X i k ) Want k most informative features: A* = argmax IG(X A ; Y) s.t. |A| · k where IG(X A ; Y) = H(Y) - H(Y | X A ) Y “Sick” X 1 “Fever” X 2 “Rash” X 3 “Male” Naïve Bayes Model Uncertainty before knowing X A Uncertainty after knowing X A

9 Example: Submodularity of info-gain Y 1,…,Y m, X 1, …, X n discrete RVs z(A) = IG(Y; X A ) = H(Y)-H(Y | X A ) z(A) is always monotonic However, NOT always submodular Theorem [Krause & Guestrin UAI’ 05] If X i are all conditionally independent given Y, then z(A) is submodular! Y1Y1 X1X1 Y2Y2 X2X2 Y3Y3 X4X4 X3X3 Hence, greedy algorithm works! In fact, NO algorithm can do better than (1-1/e) approximation!

10 People sit a lot Activity recognition in assistive technologies Seating pressure as user interface Equipped with 1 sensor per cm 2 ! Costs $16,000!  Can we get similar accuracy with fewer, cheaper sensors? Lean forward SlouchLean left 82% accuracy on 10 postures! [Tan et al] Building a Sensing Chair [Mutlu, Krause, Forlizzi, Guestrin, Hodgins UIST ‘07]

11 How to place sensors on a chair? Sensor readings at locations V as random variables Predict posture Y using probabilistic model P(Y,V) Pick sensor locations A* µ V to minimize entropy: Possible locations V AccuracyCost Before82%$16,000  After79%$100 Placed sensors, did a user study: Similar accuracy at <1% of the cost!

12 Bounds on optimal solution [Krause et al., J Wat Res Mgt ’08] Submodularity gives data-dependent bounds on the performance of any algorithm Sensing quality z(A) Higher is better Water networks data Offline (Nemhauser) bound Data-dependent bound Greedy solution Number of sensors placed

Summary (1) Minimization of submodular functions –Submodularity and convexity –Submodular Polyhedron –Symmetric submodular functions

Summary (2) Pseudo-boolean functions –Representation (polynomial, posiform, tableau, graph cut) –Reduction to quadratic polynomial –Necessary and sufficient conditions for submodularity –Minimization of quadratic and cubic submodular functions via graph cuts –Lower bound via roof duality LP via posiform representation LP via linear relaxation Max flow via symmetric graph construction

Further reading Combinatorial algorithms for submodular (and bisubmodular) function minimization More algorithms/bounds for maximizing submodular functions Linear and semidefinite relaxations Matroids, greedoids, intersection of matroids, polymatroids and more Generalized roof duality