Previously Optimization Probability Review Inventory Models Markov Decision Processes.

Slides:

Advertisements

Similar presentations

Markov Decision Processes (MDPs) read Ch utility-based agents –goals encoded in utility function U(s), or U:S  effects of actions encoded in.

Advertisements

IE 429, Parisay, January 2003 Review of Probability and Statistics: Experiment outcome: constant, random variable Random variable: discrete, continuous.

S. D. Deshmukh OM V. Capacity Planning in Services u Matching Supply and Demand u The Service Process u Performance Measures u Causes of Waiting u Economics.

A Markov Decision Model for Determining Optimal Outpatient Scheduling Jonathan Patrick Telfer School of Management University of Ottawa.

Previously Optimization Probability Review –pdf, cdf, E, Var –Poisson, Geometric, Normal, Binomial, … Inventory Models –Newsvendor Problem –Base Stock.

1 Chapter 8 Queueing models. 2 Delay and Queueing Main source of delay Transmission (e.g., n/R) Propagation (e.g., d/c) Retransmission (e.g., in ARQ)

MGTSC 352 Lecture 23: Inventory Management Big Blue Congestion Management Introduction: Asgard Bank example Simulating a queue Types of congested systems,

Queuing Models Basic Concepts

Previously Optimization Probability Review Inventory Models Markov Decision Processes.

Previously Optimization Probability Review Inventory Models Markov Decision Processes.

Chap. 20, page 1051 Queuing Theory Arrival process Service process Queue Discipline Method to join queue IE 417, Chap 20, Jan 99.

Model Antrian By : Render, ect. Outline  Characteristics of a Waiting-Line System.  Arrival characteristics.  Waiting-Line characteristics.  Service.

KI Kunstmatige Intelligentie / RuG Markov Decision Processes AIMA, Chapter 17.

Operations Management Waiting Lines. 2 Ardavan Asef-Vaziri Dec-2010Operations Management: Waiting Lines1  Questions: Can we process the orders? How many.

1 Ardavan Asef-Vaziri Sep-09Operations Management: Waiting Lines2  Made-to-stock (MTS) operations  Product is manufactured and stocked in advance of.

Previously Optimization Probability Review Inventory Models Markov Decision Processes Queues.

Previously Optimization Probability Review Inventory Models Markov Decision Processes Queues.

SIMULATION EXAMPLES. SELECTED SIMULATION EXAMPLES 4 Queuing systems (Dynamic System) 4 Inventory systems (Dynamic and Static) 4 Monte-Carlo simulation.

Agenda Duality Geometric Picture Piecewise linear functions.

Previously Optimization Probability Review Inventory Models Markov Decision Processes.

Agenda Office Hours: Friday 2-3 Final: Friday 4pm - Sat. 4pm.

Previously Optimization Probability Review Inventory Models Markov Decision Processes Queues.

Agenda Office Hours: –M 4-5, W 5-6, F 2-3 Wednesday: case discussion –1 page memo per group Friday: another case? Today: –Final review –Will post another.

Simulating Single server queuing models. Consider the following sequence of activities that each customer undergoes: 1.Customer arrives 2.Customer waits.

Previously in IEMS 310… Notation of optimization problems Linear Programs –Sensitivity Analysis / Duality –Assignment and Network Flow Problems Tricks:

Previously Optimization Probability Review Inventory Models Markov Decision Processes Queues.

1 Stochastic Modeling for Clinical Scheduling by Ji Lin Reference: Muthuraman, K., and Lawley, M. A Stochastic Overbooking Model for Outpatient Clinical.

Queuing. Elements of Waiting Lines  Population –Source of customers Infinite or finite.

1 TCOM 501: Networking Theory & Fundamentals Lectures 9 & 10 M/G/1 Queue Prof. Yannis A. Korilis.

Reinforcement Learning: Learning algorithms Yishay Mansour Tel-Aviv University.

1 Ardavan Asef-Vaziri Sep-09Operations Management: Waiting Lines3  Terminology: The characteristics of a queuing system is captured by five parameters:

Previously in IEMS 310… Notation of optimization problems Linear Programs Sensitivity Analysis Duality Piecewise linear functions Assignment Problems.

Limiting probabilities. The limiting probabilities P j exist if (a) all states of the Markov chain communicate (i.e., starting in state i, there is.

Eurecom, Sophia-Antipolis Thrasyvoulos Spyropoulos / Introduction to Continuous-Time Markov Chains and Queueing Theory.

A Somewhat Odd Service Process (Chapters 1-6)

1 1 Slide © 2005 Thomson/South-Western Chapter 12, Part A Waiting Line Models n Structure of a Waiting Line System n Queuing Systems n Queuing System Input.

1 1 © 2003 Thomson  /South-Western Slide Slides Prepared by JOHN S. LOUCKS St. Edward’s University.

4/11: Queuing Models Collect homework, roll call Queuing Theory, Situations Single-Channel Waiting Line System –Distribution of arrivals –Distribution.

ECES 741: Stochastic Decision & Control Processes – Chapter 1: The DP Algorithm 31 Alternative System Description If all w k are given initially as Then,

Giapetto's Woodcarving: The LP Model

Queueing Theory What is a queue? Examples of queues: Grocery store checkout Fast food (McDonalds – vs- Wendy’s) Hospital Emergency rooms Machines waiting.

Spreadsheet Models for Managers: Session 12 12/1 Copyright © Richard Brenner Spreadsheet Models for Managers Session 12 Service Systems Single-Server.

1 Queuing Systems (2). Queueing Models (Henry C. Co)2 Queuing Analysis Cost of service capacity Cost of customers waiting Cost Service capacity Total.

ENGM 732 Queuing Applications. Motivation Idea: We want to minimize the total cost of a queuing system Let SC = cost of service WC = cost of waiting TC.

OPSM 301: Operations Management Session 19: Flow variability Koç University Zeynep Aksin

1 1 Slide © 2005 Thomson/South-Western MANAGMENT SCIENCE n Chapter 1: Introduction Problem solving and decision making; quantitative analysis and decision.

Why Wait?!? Bryan Gorney Joe Walker Dave Mertz Josh Staidl Matt Boche.

1 1 Slide © 2011 Cengage Learning. All Rights Reserved. May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole.

Computer Networking Queueing (A Summary from Appendix A) Dr Sandra I. Woolley.

1 1 Slide © 2009 South-Western, a part of Cengage Learning Slides by John Loucks St. Edward’s University.

Review 3-25 In 2003, the average stock price for companies making up the S&P 500 was $30 per share, with a standard deviation of $8.20. Assume stock prices.

Structure of a Waiting Line System Queuing theory is the study of waiting lines Four characteristics of a queuing system: –The manner in which customers.

1 Chapter 4 Single Stage Produce-to-Stock Better customer service Lower manufacturing costs (setups) More uniform utilization.

Waiting Line Theory Akhid Yulianto, SE, MSc (log).

1 1 Slide Chapter 12 Waiting Line Models n The Structure of a Waiting Line System n Queuing Systems n Queuing System Input Characteristics n Queuing System.

SIMULATION EXAMPLES. Monte-Carlo (Static) Simulation Estimating profit on a sale promotion Estimating profit on a sale promotion Estimating profit on.

Asynchronous Control for Coupled Markov Decision Systems Michael J. Neely University of Southern California Information Theory Workshop (ITW) Lausanne,

Mohammad Khalily Islamic Azad University.  Usually buffer size is finite  Interarrival time and service times are independent  State of the system.

Queuing Theory Simulation & Modeling.

Lecture 20 Review of ISM 206 Optimization Theory and Applications.

ETM 607 – Spreadsheet Simulations

 Porter Airlines are the regional airlines of the Toronto, Canada  Head office of Porter Airlines is in Billy Bishop Toronto City Airport  Porter Airlines.

Demo on Queuing Concepts

سرطان الثدي Breast Cancer

Birth-Death Process Birth – arrival of a customer to the system

Queuing Applications.

On average, there are 3(4.42) =13.26 claims waiting be processed.

Find the limit {image} ,024 2,160 -1,

VIRTUE MARYLEE MUGURACHANI QUEING THEORY BIRTH and DEATH.

SIMULATION EXAMPLES QUEUEING SYSTEMS.

Presentation transcript:

Previously Optimization Probability Review Inventory Models Markov Decision Processes

Agenda Hwk Projects Markov Decision Processes Queues

Markov Decision Processes (MDP) States i=1,…,n Possible actions in each state Reward R(i,k) of doing action k in state i Law of motion: P(j | i,k) probability of moving i  j after doing action k

MDP f(i) = largest expected current + future profit if currently in state i f(i,k) = largest expected current+future profit if currently in state i, will do action k f(i) = max k f(i,k) f(i,k) = R(i,k) + ∑ j P(j|i,k) f(j) f(i) = max k [R(i,k) + ∑ j P(j|i,k) f(j)]

MDP as LP f(i) = max k [R(i,k) + ∑ j P(j|i,k) f(j)] Idea: f(i) decision variables piecewise linear function min ∑ j f(i) s.t. f(i) ≥ R(i,k) + ∑ j P(j|i,k) f(j) for all i,k

MDP Examples Breast cancer screening Stock options Airline ticket pricing Elevator scheduling Reservoir management

Queues (Ch 14) Queue = waiting line the “system” image from

Examples Airport security Customer service line Checkout Doctor’s office ER Canada: scheduling operations Elevators

Performance Measures T time in system T q waiting time (time in queue) N #customers in system N q #customers in queue system arrivals departures queue servers W = E[T] W q = E[T q ] L = E[N] L q = E[N q ]  fraction of time servers are busy (utilization)

Randomness is Key arrivals every 15 min (not random) processing times random with mean of 13 min (exponential random variable) waiting time customer #