Dynamic Programming Rahul Mohare Faculty Datta Meghe Institute of Management Studies

Introduction In mathematics and computer science, dynamic programming is a method for solving complex problems by breaking them down into simpler subproblems. It is applicable to problems exhibiting the properties of overlapping subproblems which are only slightly smaller

The key idea behind dynamic programming is quite simple. In general, to solve a given problem, we need to solve different parts of the problem (subproblems), then combine the solutions of the subproblems to reach an overall solution. The dynamic programming approach seeks to solve each subproblem only once, thus reducing the number of computations. This is especially useful when the number of repeating subproblems is exponentially large.

Characteristics of Dynamic Programming Applications Characteristic 1 ◦ The problem can be divided into stages with a decision required at each stage. Characteristic 2 ◦ Each stage has a number of states associated with it.  By a state, we mean the information that is needed at any stage to make an optimal decision. Characteristic 3 ◦ The decision chosen at any stage describes how the state at the current stage is transformed into the state at the next stage.

Characteristic 4 ◦ Given the current state, the optimal decision for each of the remaining stages must not depend on previously reached states or previously chosen decisions.  This idea is known as the principle of optimality. Characteristic 5 ◦ If the states for the problem have been classified into on of T stages, there must be a recursion that related the cost or reward earned during stages t, t+1, …., T to the cost or reward earned from stages t+1, t+2, …. T.

Resource-Allocation Problems Resource-allocation problems, in which limited resources must be allocated among several activities, are often solved by dynamic programming. To use linear programming to do resource allocation, three assumptions must be made: ◦ Assumption 1 : The amount of a resource assigned to an activity may be any non negative number. ◦ Assumption 2 : The benefit obtained from each activity is proportional to the amount of the resource assigned to the activity.

◦ Assumption 3: The benefit obtained from more than one activity is the sum of the benefits obtained from the individual activities. Even if assumptions 1 and 2 do not hold, dynamic programming can be used to solve resource-allocation problems efficiently when assumption 3 is valid and when the amount of the resource allocated to each activity is a member of a finite set.

Generalized Resource Allocation Problem The problem of determining the allocation of resources that maximizes total benefit subject to the limited resource availability may be written as where x t must be a member of {0,1,2,…}.

To solve this by dynamic programming, define f t (d) to be the maximum benefit that can be obtained from activities t, t+1,…, T if d unites of the resource may be allocated to activities t, t+1,…, T. We may generalize the recursions to this situation by writing f T +1(d) = 0 for all d where x t must be a non-negative integer satisfying g t (x t )≤ d.

Case-1 Capital Budgeting A XYZ Corporation is entertaining proposals from its three plants for possible expansion of facilities. This corporation is budgeting $5 Million for allocation to all three plants. Each plant is requested to submit its proposals giving total cost (c) ant total revenue (R) for each proposal. Following table summarizes the cost and revenues ProposalsPlant-1Plant-2Plant-3 C1R1C2R2C3R How allocation can be done ?

Some thing to remember… Stage: A stage in DP is defined as the portion of the problem that possess a set of mutually exclusive alternatives from which the best alternative is to be selected State: A state is normally defined to reflect the status of the constraints that bind all the stages together

In our Capital Budgeting Example Each plant defines a stage with the first, second and third stages having 3, 4 and 2 alternatives We define the states for the stages 1,2 and 3 as follows X1= amount of capital allocated to stage 1 X2= amount of capital allocated to stage 1 & 2 X3= amount of capital allocated to stage 1,2 & 3

The following table summarizes the conditional decisions for stage-1 If Available Capital X1 equals to Then, the resulting Optimal Proposal is And the total revenue of stage 1 is

X1=X2-CAPITAL allotted for stage 2 only Best revenue Stage2+stage1 Total Revenue 0 = = = = = = = = = = = = = = = conditional decisions for stage-II

Stage 3 Computations By computing all the answers for the three stages the optimal solution calls for selecting the Proposal 2 for plant 1 Proposal 4 for plant 2 Proposal 1 for plant 3 The total cost is 5 and optimal revenue is $17 millions

Dynamic Vs Linear Programming Dynamic ProgrammingLinear Programming dynamic programming is a method for solving complex problems by breaking them down into simpler sub-problems. It is applicable to problems exhibiting the properties of Overlapping sub-problems which are only slightly smaller Linear programming (LP, or linear optimization) is a mathematical method for determining a way to achieve the best outcome (such as maximum profit or lowest cost) in a given mathematical model for some list of requirements represented as linear relationships The term dynamic programming was originally used in the 1940s by Richard Bellman to describe the process of solving problems developed by leonid Kantorovich, a Russian mathematician, in 1939 Only few methods available for solving dynamic programming problem Forward recursive approach Backward recursive approch Many tools available to solve the problem Graphical Method Simplex Method Two-Phase simplex method Big-M Method DP can be used in the following areas  Molecular biology  File Compression  Screen redisplay  Matrix chain multiplication LPP can be used in the following application Product mix Make-buy Media selection Marketing research Portfolio selection Shipping & transportation Multiperiod scheduling Number of stages decide the optimal solutionDecision variables decide the optimal solutions

Thank You