Dynamic Programming and Applications Reservoir Operation Water Resources Planning and Management: M4L5 D Nagesh Kumar, IISc

Objectives To develop the steady state operational policy of a single reservoir To develop backward recursive equations for this operational policy To demonstrate the method using a numerical example Water Resources Planning and Management: M4L5 D Nagesh Kumar, IISc

Reservoir Operation – Steady State Policy Consider a single reservoir receiving inflow it and making releases rt for each time period t Maximum capacity of the reservoir is K Reservoir receives benefits from hydropower generation, irrigation, recreation etc Optimization problem: Find the sequence of releases to be made from the reservoir that maximizes the total net benefits St : initial storage for time period t St+1 : final storage for time period t Net benefits can be expressed as a function of rt, St and St+1 as Water Resources Planning and Management: M4L5 D Nagesh Kumar, IISc

Reservoir Operation – Steady State Policy… Let there be T periods in a year, then the objective function is to maximize the total net benefits from all periods Maximize The constraints are Continuity Equation after neglecting all the minor losses and assuming no overflow Capacity constraint Water Resources Planning and Management: M4L5 D Nagesh Kumar, IISc

Steady State Policy : Backward Recursion Take stages as the time periods and the states as the storage volumes Assume that there are T periods in a year To find the steady state policy, select a period in a particular year, after which it is assumed that the reservoir is no longer useful (maximum lifetime of the reservoir) Usually in almost all problems, the last period T is taken as the terminal period At this stage, the optimal release will be independent of the inflow and also the net benefit. Water Resources Planning and Management: M4L5 D Nagesh Kumar, IISc

Steady State Policy : Backward Recursion… Consider the terminal period as T of a particular year after which reservoir is no longer useful Solving this problem in a backward recursion method, let t represents the period in a year from T to 1 and n represents the periods remaining from t till end t will take values starting from T, decreasing to 1 (which will complete one year) and then again taking a value of T and repeating the values Value of n starts from 1 (while considering the Tth period of last year) and while moving backwards its value keeps on increasing i.e. at the beginning of the last year, the value of n = T and at the beginning of second last year its value will be equal to 2T and so on. Water Resources Planning and Management: M4L5 D Nagesh Kumar, IISc

Steady State Policy : Backward Recursion… Starting from the last period T of a particular year, there is only one period remaining The maximum net benefit for this last period is This is solved for all ST values from 0 to K Water Resources Planning and Management: M4L5 D Nagesh Kumar, IISc

Steady State Policy : Backward Recursion… Considering the last two stages together, the objective function is ST-1 also ranges from 0 to K In general, for a period t with n periods remaining, the function can be written as where the index t decreases from T to 1 and then takes the value T again and the cycle repeats for another year the index n starts from 1 and increases at each successive stage Water Resources Planning and Management: M4L5 D Nagesh Kumar, IISc

Steady State Policy : Backward Recursion… The cycle is repeated till the optimum values of rt for an initial storage St will be the same as the corresponding rt and St of previous year Such a solution is called stationary solution Maximum net benefit is the difference of and for any St and t. Water Resources Planning and Management: M4L5 D Nagesh Kumar, IISc

Steady State Policy : Numerical Example Consider a reservoir for which the desirable constant storage is 20 units and the constant release is 25 units. Capacity of the reservoir is 30 units. Inflows for three seasons are 10, 50 and 20 units. Optimization problem: To find the optimum rt and St that minimizes the total squared deviation from the release and storage targets given. Thus, the objective function is Minimize Water Resources Planning and Management: M4L5 D Nagesh Kumar, IISc

Steady State Policy : Numerical Example… St take the discrete values of 0, 10, 20, 30 and rt take the values of 10, 20, 30, 40 Solution: Backward recursion is used to solve the problem Consider a year after which the reservoir is no longer useful. Expressing the problem as a sequential process as shown in the figure below Water Resources Planning and Management: M4L5 D Nagesh Kumar, IISc

Steady State Policy : Numerical Example… Considering the last period for which t = 3 and n = 1, the optimization function is Minimize Inflow for 3rd season, I3 = 20 units and capacity of the reservoir, K = 30 units The release constraints can be expressed as and Water Resources Planning and Management: M4L5 D Nagesh Kumar, IISc

Steady State Policy : Numerical Example… Computation for the first subproblem (n = 1) Water Resources Planning and Management: M4L5 D Nagesh Kumar, IISc

Steady State Policy : Numerical Example… Considering the last two periods (n =2), the optimization function is Inflow for 2nd season, I2 = 50 units Release constraints can be expressed as and While computing the optimal releases for S2=30, Since r2 can take values only of 10, 20, 30 and 40 only, the release cannot be made for S2=30. Water Resources Planning and Management: M4L5 D Nagesh Kumar, IISc

Steady State Policy : Numerical Example… Computation for the second subproblem (n = 2) Water Resources Planning and Management: M4L5 D Nagesh Kumar, IISc

Steady State Policy : Numerical Example… The same procedure is repeated for all stages till n = 7. The summarized solution for this problem is given in the table below Solution for the last year Water Resources Planning and Management: M4L5 D Nagesh Kumar, IISc

Steady State Policy : Numerical Example… Solution for the second last year Water Resources Planning and Management: M4L5 D Nagesh Kumar, IISc

Steady State Policy : Numerical Example… Solution for the last period of third last year Water Resources Planning and Management: M4L5 D Nagesh Kumar, IISc

Steady State Policy : Numerical Example… At this stage, the value of r3* at n = 7 and n = 4 are exactly the same. Also the difference is same for all St This value is the minimum total squared deviations from the target release and storage Stationary optimal policy obtained is shown Water Resources Planning and Management: M4L5 D Nagesh Kumar, IISc

Steady State Policy: Limitations Assumption in dynamic programming: Decisions made at one stage is dependent only on the state variable and is independent of the decisions taken in other stages Therefore dynamic programming will not be an appropriate technique where decisions are dependent on the earlier decisions. Water Resources Planning and Management: M4L5 D Nagesh Kumar, IISc

Bibliography / Further Reading Bellman, R., Dynamic Programming, Princeton University Press, Princeton, N.J, 1957. Hillier F.S. and G.J. Lieberman, Operations Research, CBS Publishers & Distributors, New Delhi, 1987. Loucks, D.P., J.R. Stedinger, and D.A. Haith, Water Resources Systems Planning and Analysis, Prentice-Hall, N.J., 1981. Rao S.S., Engineering Optimization – Theory and Practice, Fourth Edition, John Wiley and Sons, 2009. Taha H.A., Operations Research – An Introduction, 8th edition, Pearson Education India, 2008. Vedula S., and P.P. Mujumdar, Water Resources Systems: Modelling Techniques and Analysis, Tata McGraw Hill, New Delhi, 2005. Water Resources Planning and Management: M4L5 D Nagesh Kumar, IISc

Thank You Water Resources Planning and Management: M4L5 D Nagesh Kumar, IISc