Part 1 Overview, introduction, examples What is Operations Research? What is Optimization What is Sequential Decision Making? What is Dynamic Programming? Examples, please!

Perspective Universe Applied Maths OR Optimization Sequential Decision Making

What is Operations Research? A branch of applied mathematics dealing with the Science of Decision Making Also known as Management Science One of the streams at the Maths & Stats Department More information at Local chapter

What is optimization? Extensive literature (books, journals, encyclopedia) Education and training programs Jobs Commercial software (eg. Excel) Consulting services Conferences and workshops Meeting place: e-optimization.com

Convention Universal Convention for the description of optimisation problems opt Objective (dvs) dvs Subject to: Constraints (dvs) Opt: either min or max. Objective: an expression (function) quantifying how good/bad our decisions are. dvs: decision variables Constraints: Restrictions on the decision variables.

Help Desk opt Objective (dvs) dvs Subject to: Constraints (dvs) Tip Usually it is best to build the model in the following order: –dvs (Decision variables) –Objective –Opt –Constraints But, ….. do not be dogmatic!

Naïve Example English Version Given:The value of the perimeter of a rectangle Task: Determine the width and height of the rectangle that will make its area as large as possible Maths Version Which one do you like better?

Help Desk Formally Opt = max dvs: w,h Objective: w x h Constraint: 2(w + h) = p

Help Desk h w Perimeter = 2(w+h) Area = w x h

Beware of the distinction between Problem Formulation Describing what the problem is all about Problem Solving Constructing a solution to a given problem Most students under estimate the difficulties encountered at the formulation stage

Naïve Example English Version Given:The value of the perimeter of a rectangle Task: Determine the width and height of the rectangle that will make its area as large as possible Maths Version This is the (maths) model

Observations There are infinitely many feasible solutions: Example 2(w + h) = p = 6 w = 3 - h/2 We can let h take value in the range (0,3).

Observations Not difficult to show that the optimal solution is I am a square Prove this!

Strong Operations Research Flavour Warning Parental Guidance is required

Role of Models in Problem Solving Realisation Informal Description Formal Model SolutionWorking System Modelling Analysis Implementation Monitoring mathematical ^ Don’t panic!

In Real-life Realisation Informal Description Formal Model SolutionWorking System Modelling Analysis Implementation Monitoring

We shall focus on Realisation Informal Description Formal Model Solution Working System Modelling Analysis Implementation Monitoring Given

Beware of the distinction between Problem Formulation Describing what the problem is all about Problem Solving Constructing a solution to a given problem Most students under estimate the difficulties encountered at the formulation stage

Example (Knapsack Problem) English Version Given: A container and a collection of items having known values and volumes. Task: Determine what is the sub-collection of items of maximum total value that will fit into the container. Maths Version ?????

5 minutes English Version Given: A container and a collection of items having known values and volumes. Task: Determine what is the sub-collection of items of maximum value that will fit into the container. Maths Version Help!

What is a Sequential Decision Problem ? A problem (typically an optimization problem) representing a sequential decision making situation. Example: Knapsack problem. Note the distinction between the problem itself and the formulation given to it.

What is a Dynamic Programming? Short version A paradigm for the formulation, analysis and solution for sequential decision problems. Answer Long version Wait and see …

Difficult to teach Difficult to learn Usually done “by example” Important Facts about DP

DP Approach to Problem Solving Step 1: Transform your problem into a family of related problems Step 2: Derive a functional equation relating the solutions to these problems to each other Step 3: Solve the functional equation Step 4: Recover from the functional equation a solution to your initial problem

Trouble is, This is easier said than done The Art of DP hence

Most Famous Example (Shortest Path Problem) Given: a set of nodes (cities) and the lengths of the arcs connecting them Task: Find the shortest path from some given node to another given node

Step 1: Create a Family of Related Problems Suppose we are given say m cities and have to determine the shortest path from city s to city d. Let f(j) := shortest distance from city s to city j, j=1,2,3,…,m. Note: we now have m problems.

Step 2: Deriving a Functional Equation From the definition of f(j) it follows that f(j) = t(k,j) + f(k) for some immediate predecessor k of j. Hence, where t denotes the distance matrix and P(j) denotes the set of immediate predecessors of node j.

Example f(8) =

Step 3: Solve the FE Can be –Very easy –Easy –Difficult –Very difficult –Impossible

Example (naïve) D E AB C 1 2 f(A) = 0 (by definition?) f(B) = t(A,B) + f(A) = = 3 f(C) = t(B,C) + f(B) = = 4 f(D) = t(B,D) + f(B) = = 4 f(E) = min {t(C,E) + f(C),t(D,E) + f(D)} = min {1+4,2+4} = 5

Step 4: Recover a solution Once the functional equation is solved, it is usually simple to recover an optimal solution to the original problem. (Hint: go in …. reverse)

Example (naïve) D E AB C 1 2 Optimal solutions to the FE (going from A to E) D E AB C 1 2 Optimal solution to the original problem (going from E to A)

So what is all the fuss about DP? The examples we have seen so far are extremely simple (in fact naïve) There are major complications

A (very) Famous Example

Task: Move pieces from left to right Rules One piece at a time Large on small is not allowed Objective ??

Analysis Given n pieces Move n-1 pieces to the intermediate position Move the bottom one to the destination Move the n-1 pieces from the intermediate position to the destination

Formalism Let S(n,x,y) := Solution to a problem involving moving n pieces from position x to position y. Thus, S(1,x,y) := Move a piece from x to y. Theorem: S(n,x,y) = S(n-1,x,~(x,y)), S(1,x,y), S(n-1,~(x,y),y) Length [S(n,x,y)] = 2 n - 1

Start

0

1

2

3

4

5

6

7

8

9

10

11

12

13

14

15

2 n - 1 = 2 4  1 = 16  1 = 15 15

2 n - 1 = 2 4  1 = 16  1 = 15 15

2 n - 1 = 2 4  1 = 16  1 = 15

2 n The Curse of Dimensionality Bellman [1957]

What’s next? A short visit to the tutOR site. Hands-on with the Knapsack Problem: –Conventional Formulation –DP functional equation