Combinatorial Optimization Under Uncertainty (Probing & Stopping-Time Algorithms) (10th Nov, 2017) Sahil Singla (Carnegie Mellon University) Thesis Committee: Manuel Blum, Anupam Gupta, Robert D. Kleinberg, R. Ravi, and Jan VondrÁk

Modeling Combinatorial Optimization Given a Finite Universe 𝐕={1,2,..,n} Given an Objective Function f : 2 𝐕 →R Given Constraints ℑ⊆ 2 𝐕 Select a Solution set S∈ℑ s.t. max/min 𝐟(S) Examples: Maximization: k elements, Matching, Max cut, Knapsack, (Bin) Packing, Orienteering, Welfare Max, SAT, Increasing Subseq, Throughput Scheduling, Submod over Matroid Minimization: MST, Vertex Cover, (Set) Covering, Steiner Network, TSP, Facility Location, Makespan Scheduling, Graph Coloring, Multiway/Sparsest/Multi cut, Clustering E.g, sum of weights E.g., matching

Modeling Uncertainty Why Consider Uncertainty? Lack of future knowledge Imperfect prediction: Noise/Errors Reaching certainty difficult/expensive Probing and Stopping-Time Models Uncertainty clears element-by-element Make decisions before all uncertainty clears Motivating examples

Example 1: Oil-Drilling Probing Set up One Oil Drill: Multiple potential locations Have Estimates on their Values: Location, size, old data Conduct Inspections to Find Exact Value Pay price per location Which Locations Should you Inspect? Goal: Maximize value of the oil drill Minimize total inspection price Similar Examples Purchasing a company Purchasing a house Price could be in different units, e.g., time

Underlying Combinatorial Problem X1 X1= 4 X2=𝟔 X2 X3 X3=𝟐 X4 X4=10 Price of 1 per-box Value = 𝟔 Price = 𝟑 Picked Select the Most Valuable Box (Given Value Distributions) Easy in the Full-Info World: Pick the Max-Value Box Probing Algorithms Can control the probing order Don’t probe all boxes: Price incurred Pick at the end More Complicated when Underlying Problem Changed k boxes, matching, forest

Example 2: Diamond-Selling Stopping-Time Sell One Diamond: Multiple potential buyers Buyers Arrive and Make a Take-it-or-Leave-it Bid Decide Immediately and Irrevocably When Should you Accept the Bid? Goal: Maximize value of the accepted bid Similar Examples Selling ad-slots Hiring a secretary Marriage partner Cannot go back to a declined bid

Underlying Combinatorial Problem X1 X1=4 X2=𝟔 X2 X3 X3=𝟐 X4 X4=10 Value = 6 Picked Select the Most Valuable Box Easy in the Full-Info World: Pick the Max-Value Box Stopping-Time Algorithms Cannot control the box order If picking, do immediately More Complicated when Underlying Problem Changed k boxes, matching, forest

Probing & Stopping-Time Models Similarities Given some estimates Uncertainty clears element-by-element Irrevocable decisions before all uncertainty clears Differences Probing Stopping-Time Can control the order Cannot control the order Pick in the end If picking, do immediately Price to finding value No price to finding value

OUTLINE Introduction Probing Algorithms (Oil-Drilling) Problems: Oil-Drilling and Diamond-Selling What is Comb Opt under Uncertainty Probing Algorithms (Oil-Drilling) Model 1: Constrained Stochastic Probing Model 2: Price of Information Stopping-Time Algorithms (Diamond-Selling) Model 1: Prophet Inequality Model 2: Secretary and Prophet-Secretary Further Directions Unknown or Correlated Distributions Beyond Probing and Stopping-Time Algorithms

OUTLINE Introduction Probing Algorithms (Oil-Drilling) Problems: Oil-Drilling and Diamond-Selling What is Comb Opt under Uncertainty Probing Algorithms (Oil-Drilling) Model 1: Constrained Stochastic Probing Model 2: Price of Information Stopping-Time Algorithms (Diamond-Selling) Model 1: Prophet Inequality Model 2: Secretary and Prophet-Secretary Further Directions Unknown or Correlated Distributions Beyond Probing and Stopping-Time Algorithms

Model 1 for Oil-Drilling Given Indep Bernoulli Distributions 𝑋 1 , 𝑋 2 , .. , 𝑋 𝑛 To Find Value, Pay a Given Probing Price 𝜋 1 , 𝜋 2 ,…, 𝜋 𝑛 Probing Constraints : Given Probing Budget 𝐵 𝑖∈𝑃𝑟𝑜𝑏𝑒𝑑 𝜋 𝑖 ≤𝐵 How to Adaptively Probe a Set 𝑃𝑟𝑜𝑏𝑒𝑑⊆{1,2,…,𝑛} Objective: Maximize Expected Value max 𝑖∈𝑃𝑟𝑜𝑏𝑒𝑑 {𝑋 𝑖 } 𝑿 𝒊 = 𝒗 𝒊 w.p. 𝒑 𝒊 𝟎 otherwise Think of time spent How to generalize?

Constrained Stochastic Probing Given Activation Probabilities: 𝑝 1 , 𝑝 2 , .. , 𝑝 𝑛 Set 𝐴 contains 𝑖 w.p. p 𝑖 Given Value Function 𝑓: 2 [𝑛] → 𝑅 ≥0 E.g., Max, Sum of Top k, Submodular, Subadditive Packing Probing Constraints E.g., Cardinality, Knapsack, Matroid, Orienteering Adaptively Find 𝑃𝑟𝑜𝑏𝑒𝑑⊆[𝑛] Satisfying Constraints Objective: Maximize Expected Value 𝑓(𝑃𝑟𝑜𝑏𝑒𝑑∩𝐴) Think of probability that location is feasible [n] A Prob

Optimal Adaptive Strategy No Yes 𝑋 1 𝑋 5 𝑋 2 𝑋 3 𝑋 4 𝑋 6 Every Root-Leaf Path Satisfies No variable appears twice Constraints (e.g., budget) Difficult to Find: Can be Exponential Sized! Want a Simple Optimal/Approximate Strategy

Optimal Non-Adaptive Strategy Select a Feasible set 𝑃𝑟𝑜𝑏𝑒𝑑⊆ 𝑛 in the Beginning to Maximize E 𝐴~𝑝 [𝑓 𝑃𝑟𝑜𝑏𝑒𝑑∩𝐴 ] Benefits: Easier to Represent: Just output the set Find: g 𝑆 = E 𝐴~𝑝 [𝑓 𝑆∩𝐴 ] is submod in Full-Info world Concern: Large Adaptivity 𝐆𝐀𝐏:= 𝐄[𝐀𝐃𝐀𝐏] 𝐄[𝐍𝐀] Example: Can probe ≤𝟐 𝑿 1 & 𝑿 2 take 𝟏 w.p. 𝟎.𝟓 each 𝑿 3 takes 𝟏𝟎 w.p. 𝟎.𝟎𝟏 𝐄[𝐀𝐃𝐀𝐏] ≈ 0.80 𝐄 𝐍𝐀 = 0.75 0.5 NO YES 𝑿 𝟏 𝑿 𝟐 𝑿 𝟑 Examples with GAP= 𝒆 𝒆−𝟏 ≈𝟏.𝟓𝟖

Main Result Thm [GNS’16,’17]: Adaptivity Gap for Constraints = Downward-Closed Function = Monotone Submodular is at most 3. ADAP NON-ADAP ALGO GAP 𝜶 𝜶.GAP Downward-Closed: If a set can be probed then also any subset E.g., At most 𝑘, Matching, Orienteering Submodular: If S⊆T then f S∪e −f S ≥f T∪e −f(T) E.g., Max, Sum of top 𝑘 Proof uses Two Ideas Random Root-Leaf Path Stem-by-stem Induction

Proof Ideas Assume Best ADAP Strategy Known Thm [GNS’17]: Adap Gap ≤𝟑 Only Existence of a “Good” NA-Path Assume Best ADAP Strategy Known 𝑿 𝟏 0.7 0.3 0.5 0.2 0.8 NO YES Random Path with ADAP Probabilities Example: Red path prob = 𝟎.𝟕∗𝟎.𝟓∗𝟎.𝟕 Here ADAP gets 𝑚𝑎𝑥{ 𝑣 2 , 𝑣 3 } Here NA gets E[𝑚𝑎𝑥{ 𝑋 1 , 𝑋 2 , 𝑋 3 }] 𝑿 𝟑 𝑿 𝟐 𝑿 𝟑 Stem-by-Stem Induction shows: E[Random Path] ≥ 𝟏 𝟑 ADAP Since NA ≥ E[Random Path] Adaptivity GAP ≤3

Constrained Stochastic Probing ADAP NON-ADAP ALGO GAP 𝜶 𝜶.GAP Question: The Adaptivity Gap for Constraints = Downward-Closed Function = Monotone Submodular is 𝒆/(𝒆−𝟏)? Question: The Adaptivity Gap for Constraints = Downward-Closed Function = Subadditive is 𝐩𝐨𝐥𝐲𝐥𝐨𝐠 𝐧 ? Subadditive: ∀S,∀𝑇, if f S∪T ≤f S +f(T)

OUTLINE Introduction Probing Algorithms (Oil-Drilling) Problems: Oil-Drilling and Diamond-Selling What is Comb Opt under Uncertainty Probing Algorithms (Oil-Drilling) Model 1: Constrained Stochastic Probing Model 2: Price of Information Stopping-Time Algorithms (Diamond-Selling) Model 1: Prophet Inequality Model 2: Secretary and Prophet-Secretary Further Directions Unknown or Correlated Distributions Beyond Probing and Stopping-Time Algorithms

Model 2 for Oil-Drilling Given Indep Bernoulli Distributions 𝑋 1 , 𝑋 2 , .. , 𝑋 𝑛 To Find Value, Pay a Given Probing Price 𝜋 1 , 𝜋 2 ,…, 𝜋 𝑛 How to Adaptively Probe a set 𝑃𝑟𝑜𝑏𝑒𝑑⊆{1,2,…,𝑛} Objective: Maximize Expected Utility max 𝑖∈𝑃𝑟𝑜𝑏𝑒𝑑 {𝑋 𝑖 } − 𝑖∈𝑃𝑟𝑜𝑏𝑒𝑑 𝜋 𝑖 Same as Weitzman’s Pandora’s Box Optimal policy: Index 𝝉 𝒊 based ordering 𝑿 𝒊 = 𝒗 𝒊 w.p. 𝒑 𝒊 𝟎 otherwise Price of Information

Price of Information Utility Maximization Problem Given distributions on values/costs: 𝑋 1 , 𝑋 2 , .. , 𝑋 𝑛 Given probing prices: 𝜋 1 , 𝜋 2 ,…, 𝜋 𝑛 Adaptively find 𝑃𝑟𝑜𝑏𝑒𝑑⊆{1,2,…,𝑛} Utility Maximization Problem Packing Constraints: ℑ max 𝑆⊆𝑃𝑟𝑜𝑏𝑒𝑑 & 𝑆 ∈ ℑ { 𝑖∈𝑆 𝑋 𝑖 } − 𝑖∈𝑃𝑟𝑜𝑏𝑒𝑑 𝜋 𝑖 Disutility Minimization Problem Covering Constraints: ℑ′ min 𝑆⊆𝑃𝑟𝑜𝑏𝑒𝑑 & 𝑆 ∈ ℑ′ { 𝑖∈𝑆 𝑋 𝑖 } + 𝑖∈𝑃𝑟𝑜𝑏𝑒𝑑 𝜋 𝑖 Price of Information Think of Max Weight Forest Examples Max Matroid Basis Max Wt Matching Knapsack Examples Min Matroid Basis Vertex/Set Cover Feedback Vertex Set Facility Location Prize-Collecting Steiner Tree Think of Min-cost Spanning Tree

Main Result ThM [S’18]: For any Packing/Covering Problem, an 𝜶–approx Frugal alg in the Free-Info World implies an 𝜶–approx strategy in the PoI World. Think of Max Wt Matching with 𝛼=2 Problem Approx Ratio Max/Min Matroid Basis 1 Max Wt Matching 2 Max 𝑘-system 𝑘 Max Knapsack Min Vertex-/Set-Cover min{f,log n} Min Facility Location 1.861 Min Prize Collecting Steiner Tree 3 Feedback Vertex Set O(log n)

Constrained Price of Information Given distributions on values: 𝑋 1 , 𝑋 2 , .. , 𝑋 𝑛 Given probing prices: 𝜋 1 , 𝜋 2 ,…, 𝜋 𝑛 Adaptively find 𝑃𝑟𝑜𝑏𝑒𝑑⊆{1,2,…,𝑛} Constraint: Given Price Budget 𝐵 𝑖∈𝑃𝑟𝑜𝑏𝑒𝑑 𝜋 𝑖 ≤𝐵 Maximize Expected Utility max 𝑖∈𝑃𝑟𝑜𝑏𝑒𝑑 {𝑋 𝑖 } − 𝑖∈𝑃𝑟𝑜𝑏𝑒𝑑 𝜋 𝑖 ThM [S’18]: 𝑶(𝟏) Approx for Constrained PoI. Use small Adaptivity Gap Use a Frugal Algorithm

OUTLINE Introduction Probing Algorithms (Oil-Drilling) Problems: Oil-Drilling and Diamond-Selling What is Comb Opt under Uncertainty Probing Algorithms (Oil-Drilling) Model 1: Constrained Stochastic Probing Model 2: Price of Information Stopping-Time Algorithms (Diamond-Selling) Model 1: Prophet Inequality Model 2: Secretary and Prophet-Secretary Further Directions Unknown or Correlated Distributions Beyond Probing and Stopping-Time Algorithms

Model 1 for Diamond-Selling Single Item Prophet Inequality Item value distributions known & independent Items arrive in adversarial order Immediately & Irrevocably: compare to 𝐄[𝐌𝐚𝐱] Cannot Beat ½-Approx Let 𝑿 𝟏 =1 w.p. 1 & let 𝑿 2 = 1/𝜖 w.p. 𝜖 0 otherwise Picked X1=0.3 X1~Unif(0,1) X2=0.6 X2~Exp(2) X3=0.2 X3∼0.2 X4~Unif(0,1) X4=0.9

Single Item ½-Prophet Inequality Thm [KW’12]: Threshold 𝑇=½⋅E[Max] gives ½-approx. PROOF: E Alg =𝑝⋅𝑇+E[ Alg−𝑇 + ] , where 𝑝=Pr⁡[Max≥𝑇] Observe, E[Max]≤ 𝑇+E[ Max−𝑇 + ] ≤𝑇+ 𝑖 E [ 𝑋 𝑖 −𝑇 + ] . And E Alg−𝑇 + = Pr Nothing before 𝑖 ⋅ E 𝑋 𝑖 −𝑇 + ≥ 1−𝑝 ⋅∑E 𝑋 𝑖 −𝑇 + ≥ 1−𝑝 ⋅𝑇. Hence, E Alg =𝑝⋅𝑇+E[ Alg−𝑇 + ]≥𝑇. Value = Revenue + Utility Q.E.D.

Combinatorial Prophet Inequality Thm [KW’12]: ∃½ Prophet Inequalities for Constraints = Matroid Function = Additive Thm [RS’17]: ∃𝛀(𝟏) Prophet Inequalities for Constraints = Matroid Function = Non-Negative Submodular THM [RS’17]: ∃𝛀(𝟏/(𝐥𝐨𝐠 𝐧⋅𝐥𝐨 𝐠 𝟐 𝐫)) Prophet-Inequalities for Constraints = Downward-Closed Function = Non-Negative Monotone Subadditive This result is Information Theoretic.

OUTLINE Introduction Probing Algorithms (Oil-Drilling) Problems: Oil-Drilling and Diamond-Selling What is Comb Opt under Uncertainty Probing Algorithms (Oil-Drilling) Model 1: Constrained Stochastic Probing Model 2: Price of Information Stopping-Time Algorithms (Diamond-Selling) Model 1: Prophet Inequality Model 2: Secretary and Prophet-Secretary Further Directions Unknown or Correlated Distributions Beyond Probing and Stopping-Time Algorithms

Model 2 (and 3) for Diamond-Selling Single Item Secretary Problem Item values unknown Items arrive in a uniformly random order Immediately & Irrevocably: compare to 𝐌𝐚𝐱 Can get 𝟏 𝒆 -approx and this is tight Single Item Prophet-Secretary Item value distributions known & independent Immediately & Irrevocably: compare to 𝐄[𝐌𝐚𝐱] Can we beat 1 2 -approx?

Single Item (1− 𝟏 𝒆 )-Prophet-Secretary THM [EHKS’18]: Threshold T 𝜏 =(1− e 𝜏−1 )⋅E[Max] gives (1− 1 𝑒 )-approx. PROOF: ∀𝜏, E[Max]≤ 𝑇 𝜏 +E[ Max− 𝑇 𝜏 + ] ≤ 𝑇 𝜏 +∑E [ 𝑋 𝑖 − 𝑇 𝜏 + ]. Now, E Rev =− 𝜏=0 1 𝑇 𝜏 ⋅𝑑𝑞 𝜏 , where 𝑞 𝜏 =Pr⁡[Nothing before 𝜏] and E Utility =∑E Utility i ≥ 𝜏=0 1 𝑞 𝜏 ⋅ ∑ 𝑋 𝑖 − 𝑇 𝜏 + 𝑑𝜏 ≥ 𝜏=0 1 𝑞 𝜏 ⋅ E Max − 𝑇 𝜏 𝑑𝜏 . Hence, E Alg =E Rev +E Util ≥ 1− 1 𝑒 ⋅E[Max] Value = Revenue + Utility Q.E.D.

Matroid Prophet-Secretary THM [EHKS’18]: ∃(1− 𝟏 𝒆 )-approx Prophet-Secretary for Constraints = Matroid Function = Additive Moreover, we can find it in Polynomial Time. Techniques Dynamic threshold Value = Revenue + Utility Questions Can we beat (1− 𝟏 𝒆 ) for prophet secretary? Can we get Ω(1) for matroid secretary?

OUTLINE Introduction Probing Algorithms (Oil-Drilling) Problems: Oil-Drilling and Diamond-Selling What is Comb Opt under Uncertainty Probing Algorithms (Oil-Drilling) Model 1: Constrained Stochastic Probing Model 2: Price of Information Stopping-Time Algorithms (Diamond-Selling) Model 1: Prophet Inequality Model 2: Secretary and Prophet-Secretary Further Directions Unknown or Correlated Distributions Beyond Probing and Stopping-Time Algorithms

Unknown Distrib: Sample Access Consider Prophet Inequality Underlying Probability Distributions are Unknown Can Obtain Independent Samples from Distributions Goal: Maximize expected value Minimize number of samples Model 1: Two-Phase Learning How many training samples for ½−𝜖 -approx policy? Model 2: Simultaneous Exploration and Exploitation Repeatedly play the game to maximize average value

Correlated Distrib: Markov Models Consider Prophet Inequality Given 𝑛 pairs of Indep Distributions 𝑋 1 , 𝑋 2 ,…, 𝑋 𝑛 or 𝑋 1 , 𝑋 2 ,…, 𝑋 𝑛 Don’t Know in Which World Hidden Bernoulli variable 𝒉 decides, say w.p. ½ each Cannot observe 𝒉 but box values give some idea Goal: Maximize Expected Value Questions What is the optimal algorithm? What is the optimal ratio?

Beyond Probing & Stopping-Time Two-Stage Optimization Requests/Elements appear in two-stages Minimization: Costs increase in Stage-2 e.g., Steiner Tree Maximization: Elements disappear after Stage-1 e.g., Matching [LS’17] Regret Analysis Restrict OPT to only play few strategies Additive guarantees w.r.t. OPT e.g., Scheduling problems to minimize weighted completion-time or makespan

Summary Questions? Probing Algorithms Stopping-Time Algorithms Constrained Stochastic Probing: Adaptivity Gaps Price of Information: Frugal Algorithms Stopping-Time Algorithms Prophet Inequality: Value = Utility + Revenue Secretary Problem: Divide & Conquer: Add Constraints Further Directions Unknown/Correlated Distributions Two-Stage Optimization and Regret Analysis Questions?

References S. Ehsani, T. Kesselheim, M.T. Hajiaghayi, and S. Singla. `Prophet Secretary for Matroids and Combinatorial Auctions’. SODA’18. A. Gupta, H. Jiang, Z. Scully, and S. Singla. `Markovian Price of Information' . Under Preparation. A. Gupta, V. Nagarajan, and S. Singla. `Adaptivity Gaps for Stochastic Probing: Submodular and XOS Functions' . SODA’17. A. Gupta, V. Nagarajan, and S. Singla. `Algorithms and Adaptivity Gaps for Stochastic Probing' . SODA’16. G. P. Guruganesh and S. Singla. `Online Matroid Intersection: Beating Half for Random Arrival’. IPCO’17. E. Lee and S. Singla. `Maximum Matching in the Online Batch-Arrival Model'. IPCO’17. A. Rubinstein and S. Singla. `Combinatorial Prophet Inequalities'. SODA’17. S. Singla. `The Price of Information in Combinatorial Optimization'. SODA’18.