Stochastic Optimization is (almost) as easy as Deterministic Optimization Chaitanya Swamy Joint work with David Shmoys done while at Cornell University

Stochastic Optimization Way of modeling uncertainty. Exact data is unavailable or expensive – data is uncertain, specified by a probability distribution. Want to make the best decisions given this uncertainty in the data. Applications in logistics, transportation models, financial instruments, network design, production planning, … Dates back to 1950’s and the work of Dantzig.

An Example Components Assembly Products Assemble To Order system Demand is a random variable. Want to decide inventory levels now anticipating future demand. Then get to know actual demand. Can adjust inventory levels after observing demand. Excess demand  can buy more stock paying more at the last minute Low demand  can stock inventory paying a holding cost

Two-Stage Recourse Model Given:Probability distribution over inputs. Stage I :Make some advance decisions – plan ahead or hedge against uncertainty. Observe the actual input scenario. Stage II :Take recourse. Can augment earlier solution paying a recourse cost. Choose stage I decisions to minimize (stage I cost) + (expected stage II recourse cost).

2-Stage Stochastic Facility Location Distribution over clients gives the set of clients to serve. client set D facility Stage I : Open some facilities in advance; pay cost f i for facility i. Stage I cost = ∑ (i opened) f i. stage I facility

2-Stage Stochastic Facility Location Distribution over clients gives the set of clients to serve. client set D facility Stage I : Open some facilities in advance; pay cost f i for facility i. Stage I cost = ∑ (i opened) f i. stage I facility How is the probability distribution on clients specified? A short (polynomial) list of possible scenarios Independent probabilities that each client exists A black box that can be sampled.

2-Stage Stochastic Facility Location Distribution over clients gives the set of clients to serve. facility Stage I : Open some facilities in advance; pay cost f i for facility i. Stage I cost = ∑ (i opened) f i. stage I facility Actual scenario A = { clients to serve}, materializes. Stage II : Can open more facilities to serve clients in A; pay cost f i A to open facility i. Assign clients in A to facilities. Stage II cost = ∑ f i A + (cost of serving clients in A). i opened in scenario A

2-Stage Stochastic Facility Location Distribution over clients gives the set of clients to serve. facility Stage I : Open some facilities in advance; pay cost f i for facility i. Stage I cost = ∑ (i opened) f i. stage I facility Actual scenario A = { clients to serve}, materializes. Stage II : Can open more facilities to serve clients in A; pay cost f i A to open facility i. Assign clients in A to facilities. Stage II cost = ∑ f i A + (cost of serving clients in A). i opened in scenario A

Want to decide which facilities to open in stage I. Goal: Minimize Total Cost = (stage I cost) + E A  D [stage II cost for A]. Two extremes: 1.Ultra-conservative: plan for “everything” and buy only in stage I. 2. Defer all decisions to stage II. Both strategies may be sub-optimal. Want to prove a worst-case guarantee. Give an algorithm that “works well” on any instance, and for any probability distribution.

Approximation Algorithm Hard to solve the problem exactly – even special cases are #P-hard. Settle for approximate solutions. Give polytime algorithm that always finds near- optimal solutions. A is a  -approximation algorithm if, A runs in polynomial time. A (I) ≤ . OPT(I) on all instances I,  is called the approximation ratio of A.

Previous Models Considered Dye, Stougie & Tomasgard: –approx. algorithm for a resource provisioning problem. –polynomial-scenario model. Ravi & Sinha (RS04): –other problems in the polynomial-scenario model. Immorlica, Karger, Minkoff & Mirrokni: –polynomial-scenario model and independent-activation model. –proportional costs: (stage II cost) = (stage I cost), e.g., f i A =. f i for each facility i, in each scenario A. Gupta, Pal, Ravi & Sinha (GPRS04): –black box model: arbitrary probability distributions. –Also need proportional costs.

Our Results Give the first approximation algorithms for 2-stage stochastic integer optimization problems –black-box model –no assumptions on costs. For some problems improve upon previous results obtained in more restricted models. Give a fully polynomial approximation scheme for a large class of 2-stage stochastic linear programs (contrast with Nesterov & Vial ‘00, Kleywegt, Shapiro & Homem De-Mello ‘0 1, Dyer, Kanan & Stougie ‘02). Give a way to “reduce” stochastic optimization problems to their deterministic versions.

Stochastic Set Cover (SSC) Universe U = {e 1, …, e n }, subsets S 1, S 2, …, S m  U, set S has weight w S. Deterministic problem: Pick a minimum weight collection of sets that covers each element. Stochastic version: Set of elements to be covered is given by a probability distribution. –choose some sets initially paying w S for set S –subset A  U to be covered is revealed –can pick additional sets paying w S A for set S. Minimize ( w-cost of sets picked in stage I ) + E A  U [ w A -cost of new sets picked for scenario A ].

An Integer Program For simplicity, consider w S A = W S for every scenario A. p A : probability of scenario A  U. x S : indicates if set S is picked in stage I. y A,S : indicates if set S is picked in scenario A. Minimize ∑ S  S x S + ∑ A  U p A ∑ S W S y A,S subject to, ∑ S:e  S x S + ∑ S:e  S y A,S ≥ 1 for each A  U, e  A x S, y A,S  {0, 1 }for each S, A. A Linear Program x S, y A,S ≥ 0for each S, A Exponential number of variables and exponential number of constraints.

A Rounding Theorem Assume LP can be solved in polynomial time. Suppose for the deterministic problem, we have an  -approximation algorithm wrt. the LP relaxation, i.e., A such that A (I) ≤ . (optimal LP solution for I) for every instance I. e.g., “the greedy algorithm” for set cover is a log n-approximation algorithm wrt. LP relaxation. Theorem: Can use such an  -approx. algorithm to get a 2  -approximation algorithm for stochastic set cover.

Rounding the LP Assume LP can be solved in polynomial time. Suppose we have an  -approximation algorithm wrt. the LP relaxation for the deterministic problem. Let E = {e : ∑ S:e  S x S ≥ ½}. So (2x) is a fractional set cover for the set E  can round to get an integer set cover S for E of cost ∑ S  S  S ≤  (∑ S 2  S x S ). S is the first stage decision. Let (x,y) : optimal solution with cost OPT. ∑ S:e  S x S + ∑ S:e  S y A,S ≥ 1 for each A  U, e  A  for every element e, either ∑ S:e  S x S ≥ ½ OR in each scenario A : e  A, ∑ S:e  S y A,S ≥ ½.

Sets Elements Rounding (contd.) Set in S Element in E Consider any scenario A. Elements in A  E are covered. For every e  A\E, it must be that ∑ S:e  S y A,S ≥ ½. So (2y A ) is a fractional set cover for A\E  can round to get a set cover of W-cost ≤  (∑ S 2W S y A,S ). A Using this to augment S in scenario A, expected cost ≤ ∑ S  S  S + 2 . ∑ A  U p A (∑ S W S y A,S ) ≤ 2 . OPT.

An  -approx. algorithm for deterministic problem gives a 2  -approximation guarantee for stochastic problem. In the polynomial-scenario model, gives simple polytime approximation algorithms for covering problems. 2log n-approximation for SSC. 4-approximation for stochastic vertex cover. 4-approximation for stochastic multicut on trees. Ravi & Sinha gave a log n-approximation algorithm for SSC, 2-approximation algorithm for stochastic vertex cover in the polynomial-scenario model. Rounding (contd.)

Let E = {e : ∑ S:e  S x S ≥ ½}. So (2x) is a fractional set cover for the set E  can round to get an integer set cover S of cost ∑ S  S  S ≤  (∑ S 2  S x S ). S is the first stage decision. Rounding the LP Assume LP can be solved in polynomial time. Suppose we have an  -approximation algorithm wrt. the LP relaxation for the deterministic problem. Let (x,y) : optimal solution with cost OPT. ∑ S:e  S x S + ∑ S:e  S y A,S ≥ 1 for each A  U, e  A  for every element e, either ∑ S:e  S x S ≥ ½ OR in each scenario A : e  A, ∑ S:e  S y A,S ≥ ½.

A Compact Formulation p A : probability of scenario A  U. x S : indicates if set S is picked in stage I. Minimize h(x)=∑ S  S x S + f(x) s.t. x S ≥ 0for each S (SSC-P) where, f(x)=∑ A  U p A f A (x) and f A (x)=min. ∑ S W S y A,S s.t. ∑ S:e  S y A,S ≥ 1 – ∑ S:e  S x S for each e  A y A,S ≥ 0 for each S. Equivalent to earlier LP. Each f A (x) is convex, so f(x) and h(x) are convex functions.

The General Strategy 1.Get a ( 1 +  )-optimal solution (x) to the convex program using the ellipsoid method. 2.Convert fractional solution (x) to integer solution –decouple stage I and stage II scenarios –use  -approx. algorithm for the deterministic problem to solve subproblems. Obtain a c.  -approximation algorithm for the stochastic integer problem.

If y i is infeasible, use violated inequality to chop off infeasible half-ellipsoid. The Ellipsoid Method Ellipsoid  squashed sphere Start with ball containing polytope P. y i = center of current ellipsoid. Min c. x subject to x  P. P

The Ellipsoid Method Min c. x subject to x  P. P If y i is infeasible, use violated inequality to chop off infeasible half-ellipsoid. New ellipsoid = min. volume ellipsoid containing “unchopped” half-ellipsoid. Ellipsoid  squashed sphere Start with ball containing polytope P. y i = center of current ellipsoid.

The Ellipsoid Method Min c. x subject to x  P. If y i is infeasible, use violated inequality to chop off infeasible half-ellipsoid. New ellipsoid = min. volume ellipsoid containing “unchopped” half-ellipsoid. If y i  P, use objective function cut c. x ≤ c. y i to chop off polytope, half- ellipsoid. c. x ≤ c. y i Ellipsoid  squashed sphere Start with ball containing polytope P. y i = center of current ellipsoid. P

The Ellipsoid Method Min c. x subject to x  P. Ellipsoid  squashed sphere Start with ball containing polytope P. y i = center of current ellipsoid. If y i is infeasible, use violated inequality to chop off infeasible half-ellipsoid. New ellipsoid = min. volume ellipsoid containing “unchopped” half-ellipsoid. If y i  P, use objective function cut c. x ≤ c. y i to chop off polytope, half- ellipsoid. P

The Ellipsoid Method Min c. x subject to x  P. P x 1, x 2, …, x k : points lying in P. c. x k is a close to optimal value. Ellipsoid  squashed sphere Start with ball containing polytope P. y i = center of current ellipsoid. If y i is infeasible, use violated inequality to chop off infeasible half-ellipsoid. New ellipsoid = min. volume ellipsoid containing “unchopped” half-ellipsoid. If y i  P, use objective function cut c. x ≤ c. y i to chop off polytope, half- ellipsoid. x1x1 x2x2 xkxk x*x*

Ellipsoid for Convex Optimization Min h(x) subject to x  P. P Start with ball containing polytope P. y i = center of current ellipsoid. If y i is infeasible, use violated inequality. If y i  P – how to make progress? add inequality h(x) ≤ h(y i )? Separation becomes difficult. yiyi h(x) ≤ h(y i )

Let d = subgradient at y i. use subgradient cut d. (x–y i ) ≤ 0. Generate new min. volume ellipsoid. Ellipsoid for Convex Optimization Min h(x) subject to x  P. P Start with ball containing polytope P. y i = center of current ellipsoid. If y i  P – how to make progress? d  m is a subgradient of h(.) at u, if for every v, h(v)-h(u) ≥ d. (v-u). add inequality h(x) ≤ h(y i )? Separation becomes difficult. If y i is infeasible, use violated inequality. d yiyi h(x) ≤ h(y i )

Ellipsoid for Convex Optimization Min h(x) subject to x  P. P Start with ball containing polytope P. y i = center of current ellipsoid. If y i  P – how to make progress? d  m is a subgradient of h(.) at u, if for every v, h(v)-h(u) ≥ d. (v-u). Let d = subgradient at y i. use subgradient cut d. (x–y i ) ≤ 0. Generate new min. volume ellipsoid. x 1, x 2, …, x k : points in P. Can show, min i= 1 …k h(x i ) ≤ OPT+ . x*x* x1x1 x2x2 add inequality h(x) ≤ h(y i )? Separation becomes difficult. If y i is infeasible, use violated inequality.

Let d' =  -subgradient at y i. use  -subgradient cut d'. (x–y i ) ≤ 0. Ellipsoid for Convex Optimization Min h(x) subject to x  P. P x 1, x 2, …, x k : points in P. Can show, min i= 1 …k h(x i ) ≤ OPT/( 1 -  ) + . Start with ball containing polytope P. y i = center of current ellipsoid. If y i  P – how to make progress? add inequality h(x) ≤ h(y i )? Separation becomes difficult. subgradient is difficult to compute. If y i is infeasible, use violated inequality. d'  m is a  -subgradient of h(.) at u, if  v  P, h(v)-h(u) ≥ d'. (v-u) – . h(u). d' yiyi h(x) ≤ h(y i )

Subgradients and  -subgradients Vector d is a subgradient of h(.) at u, if for every v, h(v) - h(u) ≥ d. (v-u). Vector d' is an  -subgradient of h(.) at u, if for every v  P, h(v) - h(u) ≥ d'. (v-u) – . h(u). P = { x : 0 ≤ x S ≤ 1 for each set S }. h(x) = ∑ S  S x S + ∑ A  U p A f A (x) = . x + ∑ A  U p A f A (x) Lemma: Let d be a subgradient at u, and d' be a vector such that d S –  S ≤ d' S ≤ d S for each set S. Then, d' is an  -subgradient at point u.

Getting a “nice” subgradient h(x) = . x + ∑ A  U p A f A (x) f A (x)=min. ∑ S W S y A,S s.t.∑ S:e  S y A,S ≥ 1 – ∑ S:e  S x S  e  A y A,S ≥ 0  S

Getting a “nice” subgradient h(x) = . x + ∑ A  U p A f A (x) f A (x)=min. ∑ S W S y A,S = max. ∑ e  A ( 1 – ∑ S:e  S x S ) z A,e s.t.∑ S:e  S y A,S ≥ 1 – ∑ S:e  S x S s.t. ∑ e  A  S z A,e ≤ W S  e  A  S y A,S ≥ 0  S z A,e ≥ 0  e  A

Getting a “nice” subgradient h(x) = . x + ∑ A  U p A f A (x) f A (x)=min. ∑ S W S y A,S = max. ∑ e ( 1 – ∑ S:e  S x S ) z A,e s.t.∑ S:e  S y A,S ≥ 1 – ∑ S:e  S x S s.t. ∑ e  S z A,e ≤ W S  e  A  S y A,S ≥ 0  S z A,e = 0  e  A, z A,e ≥ 0  e Consider point u  m. Let z A  optimal dual solution for A at u. So f A (u) = ∑ e ( 1 – ∑ S:e  S u S ) z A,e. For any other point v, z A is a feasible dual solution for A. So f A (v) ≥ ∑ e ( 1 – ∑ S:e  S v S ) z A,e. Get that h(v) – h(u) ≥ ∑ S (  S – ∑ A  U p A ∑ e  S z A,e )(v S – u S ) = d. (v-u) where d S =  S – ∑ A  U p A ∑ e  S z A,e. So d is a subgradient of h(.) at point u.

Computing an  -Subgradient Given point u  m. z A  optimal dual solution for A at u. Vector d with d S =  S – ∑ A  U p A ∑ e  S z A,e = ∑ A  U p A (  S – ∑ e  S z A,e ) is a subgradient at u. Goal: Get d' such that d S –  S ≤ d' S ≤ d S for each S. For each S, -W S ≤ d S ≤  S. Let = max S W S /  S. Sample once from black box to get random scenario A. Compute X S =  S – ∑ e  S z A,e for each S. E[X S ] = d S and Var[X S ] ≤ W S. 2 Sample O( 2 /  2. log(m/  )) times to compute d' such that Pr[  S, d S –  S ≤ d' S ≤ d S ] ≥ 1 -   d' is an  -subgradient at u with probability ≥ 1 - 

Putting it all together Min h(x) subject to x  P. Can compute  -subgradients. Run ellipsoid algorithm. Given y i = center of current ellipsoid. Continue with smaller ellipsoid. If y i is infeasible, use violated inequality as a cut. If y i  P use  -subgradient cut. P x1x1 x2x2 xkxk x*x* Generate points x 1, x 2, …, x k in P. Return x = argmin i= 1 …k h(x i ). Get that h(x) ≤ OPT/( 1 -  ) + .

Cannot evaluate h(.) – how to compute x = argmin i= 1 …k h(x i )? One last hurdle xkxk Will find point x in the convex hull of x 1,…, x k such that h(x) is close to min i= 1 …k h(x i ). x1x1 x2x2 x1x1 x2x2 Take points x 1 and x 2. Do bisection search using  -subgradients to find point x on x 1 –x 2 line segment with value close to min(h(x 1 ), h(x 2 )). d' :  -subgradient d'. (x-y) ≥ 0 y

One last hurdle Cannot evaluate h(.) – how to compute x = argmin i= 1 …k h(x i )? Will find point x in the convex hull of x 1,…, x k such that h(x) is close to min i= 1 …k h(x i ). xkxk x2x2 x2x2 x1x1 x1x1 Take points x 1 and x 2. Do bisection search using  -subgradients to find point x on x 1 –x 2 line segment with value close to min(h(x 1 ), h(x 2 )).

One last hurdle Cannot evaluate h(.) – how to compute x = argmin i= 1 …k h(x i )? Will find point x in the convex hull of x 1,…, x k such that h(x) is close to min i= 1 …k h(x i ). xkxk x2x2 x2x2 x1x1 x1x1 Take points x 1 and x 2. Do bisection search using  -subgradients to find point x on x 1 –x 2 line segment with value close to min(h(x 1 ), h(x 2 )).

Take points x 1 and x 2. Do bisection search using  -subgradients to find point x on x 1 –x 2 line segment with value close to min(h(x 1 ), h(x 2 )). One last hurdle Cannot evaluate h(.) – how to compute x = argmin i= 1 …k h(x i )? Will find point x in the convex hull of x 1,…, x k such that h(x) is close to min i= 1 …k h(x i ). xkxk x2x2 x1x1 x2x2 Stop when search interval is small enough. Set x = either end point of remaining segment. x1x1 x x

Take points x 1 and x 2. Do bisection search using  -subgradients to find point x on x 1 –x 2 line segment with value close to min(h(x 1 ), h(x 2 )). Iterate using x and x 3,…, x k updating x along the way. One last hurdle Cannot evaluate h(.) – how to compute x = argmin i= 1 …k h(x i )? Will find point x in the convex hull of x 1,…, x k such that h(x) is close to min i= 1 …k h(x i ). xkxk x2x2 x1x1 x

x Take points x 1 and x 2. Do bisection search using  -subgradients to find point x on x 1 –x 2 line segment with value close to min(h(x 1 ), h(x 2 )). Iterate using x and x 3,…, x k updating x along the way. One last hurdle Cannot evaluate h(.) – how to compute x = argmin i= 1 …k h(x i )? Will find point x in the convex hull of x 1,…, x k such that h(x) is close to min i= 1 …k h(x i ). xkxk x2x2 x1x1

One last hurdle Cannot evaluate h(.) – how to compute x = argmin i= 1 …k h(x i )? Will find point x in the convex hull of x 1,…, x k such that h(x) is close to min i= 1 …k h(x i ). xkxk x2x2 x1x1 Take points x 1 and x 2. Do bisection search using  -subgradients to find point x on x 1 –x 2 line segment with value close to min(h(x 1 ), h(x 2 )). Iterate using x and x 3,…, x k updating x along the way.

One last hurdle Cannot evaluate h(.) – how to compute x = argmin i= 1 …k h(x i )? Will find point x in the convex hull of x 1,…, x k such that h(x) is close to min i= 1 …k h(x i ). xkxk x2x2 x x1x1 Can show that h(x) is close to min i= 1 …k h(x i ). Take points x 1 and x 2. Do bisection search using  -subgradients to find point x on x 1 –x 2 line segment with value close to min(h(x 1 ), h(x 2 )). Iterate using x and x 3,…, x k updating x along the way.

Finally, Get solution x with h(x) close to OPT. Sample initially to detect if OPT = Ω( 1 / ) – this allows one to get a ( 1 +  ). OPT guarantee. Theorem: (SSC-P) can be solved to within a factor of ( 1 +  ) in polynomial time, with high probability. Gives a (2log n+  )-approximation algorithm for the stochastic set cover problem.

A Solvable Class of Stochastic LPs Minimize h(x)=w. x + ∑ A  U p A f A (x) s.t. x  m, x ≥ 0, x  P where f A (x)=min. w A. y A + c A. r A s.t. B A r A ≥ j A D A r A + T A y A ≥ l A – T A x y A  m, r A  n, y A ≥ 0, r A ≥ 0. Theorem: Can get a ( 1 +  )-optimal solution for this class of stochastic programs in polynomial time.

Sample Average Approximation Sample Average Approximation (SAA) method: –Sample initially N times from scenario distribution –Solve 2-stage problem estimating p A with frequency of occurrence of scenario A How large should N be? Kleywegt, Shapiro & Homem De-Mello (KSH0 1 ): –bound N by variance of a certain quantity – need not be polynomially bounded even for our class of programs. ShmoysS (recent work): –show using  -subgradients that for our class, N can be poly-bounded. Nemirovskii & Shapiro: –show that for SSC with non-scenario dependent costs, KSH0 1 gives polynomial bound on N for (preprocessing + SAA) algorithm.

Summary of Results Give the first FPRAS to solve a class of stochastic linear programs. Obtain the first approximation algorithms for 2-stage discrete stochastic problems – no assumptions on costs or probability distribution. –2log n-approx. for set cover –4-approx. for vertex cover (VC) and multicut on trees. GPRS04 gave an 8-approx. for VC under proportional costs in the black box model.

Results (contd.) –3.23-approx. for uncapacitated facility location (FL). Get constant guarantees for several variants such as FL with penalties, or soft capacities, or services. Improve upon results of GPRS04, RS04 obtained in restricted models. –( 1 +  )-approx. for multicommodity flow. Immorlica et al. gave optimal algorithm for single- commodity in polynomial “scenario” (demand) case. Give a general technique to lift deterministic guarantees to stochastic setting.

Open Questions Practical Impact: Can one use  -subgradients in other deterministic optimization methods, e.g., cutting plane methods? Interior-point algorithms? Multi-stage stochastic optimization. Characterize which deterministic problems are “easy to solve” in stochastic setting, and which problems are “hard”.

Thank You.