The Price of information in combinatorial optimization (10th January, 2018) Sahil Singla (Carnegie Mellon University)

Motivation: Setting Up an Oil Drill Set up One Oil Drill: Multiple potential sites Have Estimates on their Values: Location, size, surveys Conduct Inspections to Find Exact Value Pay price per site Which Sites Should you Inspect? Want to Maximize Max[value of inspected site] - Total inspection price Similar Examples Purchasing a company Purchasing a house

Pandora’s Box How to generalize? Maximize Expected Utility Given independent distributions on values: 𝑋 1 , 𝑋 2 , .. , 𝑋 𝑛 Given probing prices: 𝜋 1 , 𝜋 2 ,…, 𝜋 𝑛 Adaptively find 𝑃𝑟𝑜𝑏𝑒𝑑⊆{1,2,…,𝑛} Maximize Expected Utility max 𝑖∈𝑃𝑟𝑜𝑏𝑒𝑑 {𝑋 𝑖 } − 𝑖∈𝑃𝑟𝑜𝑏𝑒𝑑 𝜋 𝑖 Price of Information Price of 1 per-box X1~Unif(0,10) X1= 4 X2~Exp(0.5) X2=𝟔 X3~2 X3=𝟐 X4~Unif(0,10) X4=9 Value = 𝟔 Price = 𝟑 Value = 𝟔 Price = 𝟐 Value = 𝟐 Price = 𝟏 Picked How to generalize? Weitzman’s optimal policy

Max-/Min-Weight Spanning Tree Given distributions on edge values/costs: 𝑋 1 , 𝑋 2 , .. , 𝑋 𝑛 Given probing prices: 𝜋 1 , 𝜋 2 ,…, 𝜋 𝑛 Adaptively find 𝑃𝑟𝑜𝑏𝑒𝑑⊆{1,2,…,𝑛} Utility Maximization Problem Packing forest constraints: ℑ max 𝑆⊆𝑃𝑟𝑜𝑏𝑒𝑑 & 𝑆 ∈ ℑ { 𝑖∈𝑆 𝑋 𝑖 } − 𝑖∈𝑃𝑟𝑜𝑏𝑒𝑑 𝜋 𝑖 Disutility Minimization Problem Covering spanning-tree constraints: ℑ′ min 𝑆⊆𝑃𝑟𝑜𝑏𝑒𝑑 & 𝑆 ∈ ℑ′ { 𝑖∈𝑆 𝑋 𝑖 } + 𝑖∈𝑃𝑟𝑜𝑏𝑒𝑑 𝜋 𝑖 Price of Information

Price of Information (PoI) Given distributions on values/costs: 𝑋 1 , 𝑋 2 , .. , 𝑋 𝑛 Given probing prices: 𝜋 1 , 𝜋 2 ,…, 𝜋 𝑛 Adaptively find 𝑃𝑟𝑜𝑏𝑒𝑑⊆{1,2,…,𝑛} Utility Maximization Problem Packing constraints: ℑ max 𝑆⊆𝑃𝑟𝑜𝑏𝑒𝑑 & 𝑆 ∈ ℑ { 𝑖∈𝑆 𝑋 𝑖 } − 𝑖∈𝑃𝑟𝑜𝑏𝑒𝑑 𝜋 𝑖 Disutility Minimization Problem Covering constraints: ℑ′ min 𝑆⊆𝑃𝑟𝑜𝑏𝑒𝑑 & 𝑆 ∈ ℑ′ { 𝑖∈𝑆 𝑋 𝑖 } + 𝑖∈𝑃𝑟𝑜𝑏𝑒𝑑 𝜋 𝑖 Price of Information Examples Max Matroid Basis Maximum Matching Knapsack Examples Min Matroid Basis Vertex/Set Cover Feedback Vertex Set Facility Location Prize-Collecting Steiner Tree Semiadditive functions

Main Result Frugal ≈ “Greedy” Theorem 1: For any Packing/Covering Problem, an 𝜶–approx Frugal alg in the Free-Info World implies an 𝜶–approx strategy in the PoI World. Think of Max Wt Matching with 𝛼=2 Problem Approx Ratio Max/Min Matroid Basis 1 Max Matching 2 Max 𝑘-system 𝑘 Max Knapsack Min Vertex-/Set-Cover min{f,log n} Min Facility Location 1.861 Min Prize Collecting Steiner Tree 3 Feedback Vertex Set O(log n)

OUTLINE Pandora’s Box and Price of Information Intuitive Examples Bounding the Optimum Strategy Using a Frugal Algorithm to Design a Strategy Extensions and Conclusions

Optimal Adaptive Strategy Assume Bernoulli Variables: 𝑿 𝒊 = 𝒗 𝒊 w.p. 𝒑 𝒊 𝟎 otherwise No Yes 𝑋 1 𝑋 5 𝑋 2 𝑋 3 𝑋 4 𝑋 6 Adaptive Decision Tree Difficult to Find: Can be Exponential Sized Want Simple Optimal/Approximate Strategy Not comparing to hindsight optimum

Given Distributions : 𝑋 1 , 𝑋 2 , .. , 𝑋 𝑛 Given Probing Prices: 𝜋 1 , 𝜋 2 ,…, 𝜋 𝑛 Adaptively find 𝑃𝑟𝑜𝑏𝑒𝑑⊆{1,2,…,𝑛} Maximize Expected Utility max 𝑖∈𝑃𝑟𝑜𝑏𝑒𝑑 {𝑋 𝑖 } − 𝑖∈𝑃𝑟𝑜𝑏𝑒𝑑 𝜋 𝑖 Examples Each has price 𝟏 𝑿 𝟏 , 𝑿 𝟐 ,…, 𝑿 𝟏𝟎𝟎𝟎 = 𝟐𝟎𝟎 w.p. 𝟎.𝟎𝟏 𝟎 otherwise 𝑿 𝟏 has price 𝟏𝟗𝟖 and others have price 𝟏 𝑿 𝟏 =𝟐𝟎𝟎 w.p. 𝟏 & 𝑿 𝟐 , 𝑿 𝟑 ,…, 𝑿 𝟏𝟎𝟎𝟎 = 𝟐𝟎𝟎 w.p. 𝟎.𝟎𝟏 𝟎 otherwise Weitzman’s Index 𝝉 𝒊 is solution to 𝐸 𝑋 𝑖 − 𝜏 𝑖 + = 𝜋 𝑖 Open in decreasing index order E[Opt] ≈𝟏𝟎𝟎 Simple Stopping rule: ∀𝒊: 𝐸 𝑀𝑎𝑟𝑔𝑖𝑛𝑎𝑙(𝑖) =𝐸 (𝑋 𝑖 −𝑐𝑢𝑟𝑟𝑒𝑛𝑡 + ]≤ 𝜋 𝑖 Naïve Greedy order can be bad: 𝑎𝑟𝑔𝑚𝑎 𝑥 𝑖 (𝐸 (𝑋 𝑖 −𝑐𝑢𝑟𝑟𝑒𝑛𝑡 + ]− 𝜋 𝑖 ) How to prove?

Proof Idea Theorem 1 (Utility Maximization): Any 𝜶–approx Frugal alg in Free-Info World implies an 𝜶–approx strategy in the PoI World. Three Steps: E[Optimal Adap in PoI] ≤ E[Optimal Surrog in Free-Info] Optimal Surrog in Free-Info ≤ 𝜶 × (Frugal Alg in Free-Info) E[Frugal Alg in Free-Info] = E[Frugal Strategy in PoI] ⟹ E[Optimal Adap in PoI] ≤ 𝜶 × E[Frugal Strategy in PoI]

OUTLINE Pandora’s Box and Price of Information Intuitive Examples Bounding the Optimum Strategy Using a Frugal Algorithm to Design a Strategy Extensions and Conclusions

The Surrogate Utility Maximization Problem E[Optimal Adap in PoI] ≤ E[Optimal Surrog in Free-Info] Optimal Surrog in Free-Info ≤ 𝜶 × (Frugal Alg in Free-Info) E[Frugal Alg in Free-Info] = E[Frugal Strategy in PoI] The Surrogate Utility Maximization Problem Index 𝜏 𝑖 is solution to 𝐸 𝑋 𝑖 − 𝜏 𝑖 + = 𝜋 𝑖 Surrogate 𝑌 𝑖 = min 𝑋 𝑖 , 𝜏 𝑖 Pandora’s Box: 𝐄[𝑂𝑝𝑡]≤𝐄[max{ 𝑌 1 , 𝑌 2 ,…, 𝑌 𝑛 }] Packing Lemma: 𝐄[𝑂𝑝𝑡]≤𝐄[ max & 𝑆 ∈ ℑ ⁡{ 𝑖∈𝑆 𝑛 𝑌 𝑖 }] Example: Single box 𝑋 1 𝐸[𝑂𝑝𝑡]=𝐸 𝑋 1 − 𝜋 1 𝐸 𝑌 1 = E[min 𝑋 1 , 𝜏 1 ]=𝐸[ 𝑋 1 − 𝑋 1 − 𝜏 1 + ] =𝐸 𝑋 1 − 𝜋 1 Similar definitions for disutility minimization Better than E[𝑂𝑝𝑡]≤𝐸[max{ 𝑋 1 , 𝑋 2 ,…, 𝑋 𝑛 }]

Packing Lemma Lemma: 𝐄[𝑂𝑝𝑡]≤𝐄[ max & 𝑆 ∈ ℑ ⁡{ 𝑖∈𝑆 𝑌 𝑖 }] E[Optimal Adap in PoI] ≤ E[Optimal Surrog in Free-Info] Optimal Surrog in Free-Info ≤ 𝜶 × (Frugal Alg in Free-Info) E[Frugal Alg in Free-Info] = E[Frugal Strategy in PoI] Packing Lemma Lemma: 𝐄[𝑂𝑝𝑡]≤𝐄[ max & 𝑆 ∈ ℑ ⁡{ 𝑖∈𝑆 𝑌 𝑖 }] Proof: Let 𝑨 𝒊 and 𝟏 𝒊 denote Opt picking and Opt probing, resp. 𝐄 𝑂𝑝𝑡 = 𝐄 𝑖 ( 𝑨 𝒊 𝑋 𝑖 − 𝟏 𝒊 𝜋 𝑖 ) = 𝐄 𝑖 ( 𝑨 𝒊 𝑋 𝑖 − 𝟏 𝒊 𝑋 𝑖 − 𝜏 𝑖 + ) ≤ 𝐄 𝑖 ( 𝑨 𝒊 𝑋 𝑖 − 𝑨 𝒊 𝑋 𝑖 − 𝜏 𝑖 + ) =𝐄[ 𝑖 ( 𝑨 𝒊 𝑌 𝑖 )] ≤ 𝐸[ max & 𝑆 ∈ ℑ ⁡{ 𝑖∈𝑆 𝑌 𝑖 }] 𝐄 𝑋 𝑖 − 𝜏 𝑖 + = 𝜋 𝑖 𝑌 𝑖 = min 𝑋 𝑖 , 𝜏 𝑖 since 𝑨 𝒊 ≤ 𝟏 𝒊 Q.E.D. Note: Proof similar to [Kleinberg et al. EC’16]

OUTLINE Pandora’s Box and Price of Information Intuitive Examples Bounding the Optimum Strategy Using a Frugal Algorithm to Design a Strategy Extensions and Conclusions

Frugal Alg An ALG in Free-Info World is Frugal Packing if Examples E[Optimal Adap in PoI] ≤ E[Optimal Surrog in Free-Info] Optimal Surrog in Free-Info ≤ 𝜶 × (Frugal Alg in Free-Info) E[Frugal Alg in Free-Info] = E[Frugal Strategy in PoI] Frugal Alg An ALG in Free-Info World is Frugal Packing if ∃ marginal-value function 𝑔 𝒀 𝑀 , 𝑖, 𝑌 𝑖 ≥0 increasing in 𝑌 𝑖 In an iteration, let j have largest marginal (“best”) If j can be picked then pick j irrevocably: 𝑀=𝑀∪𝑗 Else, discard j forever Think of Max Wt Matching: 𝑔( 𝒀 𝑀 ,𝑖, 𝑌 𝑖 )= 𝑌 𝑖 Examples Matroids or Matching: 𝑔( 𝒀 𝑀 ,𝑖, 𝑌 𝑖 )= 𝑌 𝑖 Set cover: 𝑔( 𝒀 𝑀 ,𝑖, 𝑌 𝑖 )= 1 𝑌 𝑖 ( ∪ 𝑗∈𝑀∪𝑖 𝑆 𝑗 − ∪ 𝑗∈𝑀 𝑆 𝑗 ) Observations Marginal 𝑔 𝒀 𝑀 , 𝑖, 𝑌 𝑖 is independent of unseen elements Captures primal-dual algos without “cleanup” phase

Frugal Strategy Strategy Proof Idea E[Optimal Adap in PoI] ≤ E[Optimal Surrog in Free-Info] Optimal Surrog in Free-Info ≤ 𝜶 × (Frugal Alg in Free-Info) E[Frugal Alg in Free-Info] = E[Frugal Strategy in PoI] Frugal Strategy Strategy Use index for unprobed elements to compute marginal In an iteration, let j have largest marginal (“best”) If j cannot be picked, discard j forever Else, if j is unprobed then probe it otherwise, j is already probed and pick it Proof Idea Couple both worlds: Same expected change in value Although no price in Free-Info, they get lower value 𝑔( 𝒀 𝑀 ,𝑖, 𝜏 𝑖 ) 𝑌 𝑖 = min 𝑋 𝑖 , 𝜏 𝑖

Applications Problem Approx Ratio Max/Min Matroid Basis 1 Max Matching 2 Max 𝑘-system 𝑘 Max Knapsack Min Vertex-/Set-Cover min{f,log n} Min Facility Location 1.861 Min Prize Collecting Steiner Tree 3 Feedback Vertex Set O(log n)

OUTLINE Pandora’s Box and Price of Information Intuitive Examples Bounding the Optimum Strategy Using a Frugal Algorithm to Design a Strategy Extensions and Conclusions

General Functions Disutility Minimization Covering constraints ℑ′: min 𝑆⊆𝑃𝑟𝑜𝑏𝑒𝑑 & 𝑆 ∈ ℑ′ { 𝑖∈𝑆 𝑋 𝑖 } + 𝑖∈𝑃𝑟𝑜𝑏𝑒𝑑 𝜋 𝑖 More generally, min 𝑆⊆𝑃𝑟𝑜𝑏𝑒𝑑 & 𝑆 ∈ ℑ′ {𝑐𝑜𝑠𝑡(𝑆, 𝑋 )} + 𝑖∈𝑃𝑟𝑜𝑏𝑒𝑑 𝜋 𝑖 Semiadditive Functions: 𝑐𝑜𝑠𝑡 𝑆, 𝑋 = 𝑖∈𝑆 𝑋 𝑖 +ℎ(𝑆) 1 Facility location: ℎ 𝑆 = 𝑗∈𝑐𝑙𝑖𝑒𝑛𝑡𝑠 min 𝑖∈𝑆 𝑑(𝑖,𝑗) Examples Min Matroid Basis Vertex/Set Cover Feedback Vertex Set Examples Facility Location Prize-Collecting Steiner Tree ℎ: 2 𝑉 → 𝑅 ≥0 independent of 𝑋

Constrained Utility Max Consider Pandora’s Box Besides Prices, Allowed to Probe at most k items How to Adaptively probe a set 𝑃𝑟𝑜𝑏𝑒𝑑, s.t. 𝑃𝑟𝑜𝑏𝑒𝑑 ≤𝑘 Maximize Expected Utility max 𝑖∈𝑃𝑟𝑜𝑏𝑒𝑑 {𝑋 𝑖 } − 𝑖∈𝑃𝑟𝑜𝑏𝑒𝑑 𝜋 𝑖 Use Small Adaptivity Gap for Stoch Probing [GNS’17] Find NA Soln: Submod max over knapsack Convert NA soln to PoI strategy: Use Pandora’s Box Other applications: Set Probing Problem More generally, any packing constraint

Open Problems Problem 1: Any Interesting Algorithms Beyond Frugal? Min s-t cut in the PoI-World Shortest s-t Path in the PoI-World Problem 2: Any Hardness Results for Max-Matching in the PoI-World? Problem 3: Frugal Algorithms with Better Approx Factors? Feedback Vertex Set Max Matching Problem 4: How to Learn and Optimize from Samples when Probability Distributions Not Known?

Summary Questions? What is Price of Information Pandora’s Box Many packing/covering problems Frugal Algorithms Suffice OPT bounded by a random surrogate in Free-Info world Run Frugal algorithm in PoI world using index Extensions Constrained Utility Max and Set Probing Markov Chains by generalizing Gittins index policies Open Problems Beyond Frugal algos and prove Hardness results Questions?

References I. Dumitriu, P. Tetali, and P. Winkler. `On playing golf with two balls’. SIDMA’03. A. Gupta, H. Jiang, Z. Scully, and S. Singla. ` The Markovian Price of Information ' . In Preparation. A. Gupta, V. Nagarajan, and S. Singla. `Adaptivity Gaps for Stochastic Probing: Submodular and XOS Functions' . SODA’17. A. Gupta, V. Nagarajan, and S. Singla. `Algorithms and Adaptivity Gaps for Stochastic Probing' . SODA’16. R. Kleinberg, B. Waggoner, and E. Glen Weyl. `Descending Price Optimally Coordinates Search' . EC’16. S. Singla. `The Price of Information in Combinatorial Optimization'. SODA’18. M. L. Weitzman. `Optimal Search for the Best Alternative’. Econometrica’79.