1. Previously Optimization Probability Review –pdf, cdf, E, Var –Poisson, Geometric, Normal, Binomial, … Inventory Models –Newsvendor Problem –Base Stock Model

2. Agenda Projects Order Quantity Model –aka Economic Order Quantity (EOQ) Markov Decision Processes

3. Order Quantity Model Continuous review (instead of periodic) Ordering costs vs. Inventory costs Q: When to reorder? time inventory reorder times

4. Order Quantity Model (12.7) Constant demand rateA/year Inventory –No backlogging –Replenishment lead timeL years (Time between ordering more and delivery) –Order placement cost$K (Independent of order size) –Holding costH/unit/year Q: Reorder point r? Order quantity q?

5. Order Quantity Model slope = -A time inventory … order quantity q reorder point r lead time L

6. Economic Order Quantity (EOQ) r=AL time between orders =q/A orders per year = A/q ordering cost per year = KA/q holding cost per year: H(q/2) slope = -A time inventory … order quantity q reorder point r lead time L

7. Economic Order Quantity (EOQ) ordering cost per year = KA/q holding cost per year: H(q/2) total cost C(q) = KA/q + Hq/2 max C(q) s.t. q≥0 C’(q) = H/2 - KA/q 2, C’(q*)=0 q* = (2AK/H) 1/2 (cycle stock) -A time inventory … q r L

8. Summary of Inventory Models Newsvendor model Base stock model –safety stock Order quantity model –cycle stock Growth with square-root of demand 12.8 covers order quantity + uncertain demand

9. Markov Decision Processes (9.10-9.12) Junk Mail example (9.12) $1.80 to print and mail a catalog $25 profit if you buy something 5% probability of buying if new customer expected profit = -$1.80 + 5%*$25 = -$0.55 but you might be a profitable repeat customer i0123456+ p(i)0.050.400.200.100.030.010.00 p(i) probability of an order if received i catalogs since last order (i=1 means ordered from last catalog sent, i=0 means new customer)

10. Junk Mail Example Give up on customers with 6 catalogs and no orders 7 states i=0,…,6+ f(i) = largest expected current+future profit from a customer in state i i0123456+ p(i)0.050.400.200.100.030.010.00 p(i) probability of an order if received i catalogs since last order (i=1 means ordered from last catalog sent, i=0 means new customer)

11. LP Form Idea: f(i) decision variables piecewise linear function min f(0)+…+f(6) s.t. f(i) ≥ -1.80+p(i)[25+f(1)]+[1-p(i)]f(i+1) for i=1..5 f(0) ≥ -1.80+p(i)[25+f(1)] f(i) ≥ 0for all i

12. Markov Decision Processes (MDP) States i=1,…,n Possible actions in each state Reward R(i,k) of doing action k in state i Law of motion: P(j | i,k) probability of moving i  j after doing action k

13. MDP f(i) = largest expected current + future profit if currently in state i f(i,k) = largest expected current+future profit if currently in state i, will do action k f(i) = max k f(i,k) f(i,k) = R(i,k) + ∑ j P(j|i,k) f(j) f(i) = max k [R(i,k) + ∑ j P(j|i,k) f(j)]

14. MDP as LP f(i) = max k [R(i,k) + ∑ j P(j|i,k) f(j)] Idea: f(i) decision variables piecewise linear function min ∑ j f(i) s.t. f(i) ≥ R(i,k) + ∑ j P(j|i,k) f(j) for all i,k