1. Dynamic Product Assembly and Inventory Control for Maximum Profit Michael J. Neely, Longbo Huang (University of Southern California) Proc. IEEE Conf. on Decision and Control (CDC), Atlanta, GA, Dec. 2010 PDF of paper at: http://www-bcf.usc.edu/~mjneely/ Sponsored in part by the NSF Career CCF-0747525 M Raw MaterialsK Products

2. Product Assembly System M Raw MaterialsK Products Example Recipe for Product 1: [One unit of Product 1] = + Challenge: The “underflow” problem!

3. Product Assembly System M Raw MaterialsK Products Raw Material Purchasing Decisions: x m (t) = Random cost for material m on slot t. A m (t) = Amount of material m we purchase on slot t. Decision constraint: 0 ≤ A m (t) ≤ A m max. A 1 (t) A 2 (t) A 3 (t) A 4 (t) Expense(t) = ∑ m A m (t)x m (t)

4. Product Assembly System M Raw MaterialsK Products P 1 (t) Product Pricing Decisions: p k (t) = Price for product k chosen on slot t. y(t) = Demand state for slot t. D k (t) = Random demand for product k on slot t. F k (p, y) = Demand Function = E{D k (t)|p k (t)=p, y(t)=y}. A 1 (t) A 2 (t) A 3 (t) A 4 (t) Revenue(t) = ∑ k p k (t)D k (t) – Assembly Cost P 2 (t) P 3 (t)

5. About the Demand Function F() Example 1: Demand State y(t) in {High, Medium, Low, Zero}. F k (p, y) = E{D k (t)|p k (t) = p, y(t) = y}. price p k (for product k) Expected demand F k (p,y) High Medium Low

6. About the Demand Function F() Example 2: Special case of no pricing decisions. p k (t) = p k (fixed at some constant for all time). Y(t) = (Y 1 (t), …, Y K (t)) = Random demand for slot t. D k (t) = Y k (t). M Raw MaterialsK Products A 1 (t) A 2 (t) A 3 (t) A 4 (t) D 1 (t) D 2 (t) D 3 (t)

7. Problem Formulation: Every slot t, observe: Queue states Q m (t) Raw material prices p m (t) Demand state y(t). Make purchasing decisions (A m (t)) and pricing decisions (p m (t)) to maximize time average profit: Average Profit = lim t  infinity (1/t)∑ τ=0 Profit(τ) Profit(t) = Revenue(t) – Expense(t) t-1

8. Solution Strategy: Dynamic Queue Control Q m (t) θmθm Want to keep queue size high to avoid underflow. Don’t want queues too high (un-used inventory). Lyapunov Function (same as the stock market work): L(Q(t)) = ∑ m (Q m (t) – θ m ) 2

9. Control Design: Q n (t) Lyapunov Function L(Q(t)) Lyapunov Drift Δ(t) = L(Q(t+1)) – L(Q(t)) Every slot t, observe Q(t), p(t), and minimize the drift-plus-penalty*: θnθn *from stochastic network optimization theory [Georgiadis, Neely, Tassiulas 2006][Neely 2010] Δ(t) - V x Profit(t) Results in the following algorithm: (Raw Material Purchasing) Observe costs (x m (t)). A m (t) = A m max, if Vx m (t) + Q m (t) ≤ θ m A m (t) = 0, if Vx m (t) + Q m (t) > θ m

10. Control Design: Q n (t) Lyapunov Function L(Q(t)) Lyapunov Drift Δ(t) = L(Q(t+1)) – L(Q(t)) Every slot t, observe Q(t), p(t), and minimize the drift-plus-penalty*: θnθn *from stochastic network optimization theory [Georgiadis, Neely, Tassiulas 2006][Neely 2010] Δ(t) - V x Profit(t) Results in the following algorithm: (Product Pricing) Observe demand state y(t). Max: Vp k (t)F k (p k (t), y(t)) + F k (p k (t),Y(t))∑ m β mk (Q m (t) – θ m ) Subject to: 0 ≤ p k (t) ≤ p k max ( β mk = Num type m materials used for product k. )

11. Theorem (iid case): Suppose random (x m (t), y(t)) Vector is iid over slots. *Choose θ m = C 1 + C 2 V. Then: (a)For all slots t we have: μ m max ≤ Q m (t) ≤ θ m + A m max (and thus never have underflow) (b)For all slots t>0 we have: (1/t) ∑ τ=0 E{Profit(τ)} ≥ Profit opt - B/V - L(Q(0))/(Vt) (c)With prob 1 we have: *The constants C 1, C 2 depend on β mk, P k max, μ m max, A m max, and are given in the paper. *μ m max = Max departures of raw material m on one slot. t-1 lim t  infinity (1/t)∑ τ=0 Profit(τ) ≥ Profit opt – B/V

12. Theorem (iid case): Suppose random (x m (t), y(t)) vector is iid over slots. *Choose θ m = C 1 + C 2 V. Then: (a)For all slots t we have: μ m max ≤ Q m (t) ≤ θ m + A m max (and thus never have underflow) (b)For all slots t>0 we have: (1/t) ∑ τ=0 E{Profit(τ)} ≥ Profit opt - B/V - L(Q(0))/(Vt) (c)With prob 1 we have: t-1 lim t  infinity (1/t)∑ τ=0 Profit(τ) ≥ Profit opt – B/V Can also extend to the non-iid case via “universal scheduling” techniques.

13. Extensions for Multi-Hop and Recent Related Work: Multi-Hop “Processing” or “Product Assembly” Networks are of recent interest and can be used as models for manufacturing, network coding, cloud computing, data fusion. Using max-weight backpressure doesn’t work (underflow problems). Multi-Hop “Processing Networks” and the Deficit-Max-Weight Algorithm asymptotically avoids underflows for networks with some limitations [Jiang, Walrand Allerton 2009] [Jiang, Walrand, Morgan&Claypool 2010]. Recent work by [Huang, Neely, ArXiv 2010] uses the Lyapunov function L(Q(t)) = ∑ m (Q m (t) – θ m ) 2 to avoid underflows in multi-hop. Different intuition given by Huang interprets this Lyapunov function as “lifting” the Lagrange multipliers for equality constraints to ensure they are non- negative (corresponding to physical systems). This intuition can yield improved O(log 2 (V)) buffer size tradeoffs! Q n (t) θmθm

14. Conclusions: M Raw MaterialsK Products Product Assembly System with “Recipe” for each product. Dynamic Purchasing and Pricing avoids underflows. Gets O(V) worst-case queue backlog. Gets within O(1/V) of optimal time average profit.