1. Dynamic Programming and Applications
Reservoir Operation
Water Resources Planning and Management: M4L5
D Nagesh Kumar, IISc

2. Objectives
To develop the steady state operational policy of a single reservoir
To develop backward recursive equations for this operational policy
To demonstrate the method using a numerical example
Water Resources Planning and Management: M4L5
D Nagesh Kumar, IISc

3. Reservoir Operation – Steady State Policy
Consider a single reservoir receiving inflow it and making releases rt for each time period t
Maximum capacity of the reservoir is K
Reservoir receives benefits from hydropower generation, irrigation, recreation etc
Optimization problem: Find the sequence of releases to be made from the reservoir that maximizes the total net benefits
St : initial storage for time period t
St+1 : final storage for time period t
Net benefits can be expressed as a function of rt, St and St+1 as
Water Resources Planning and Management: M4L5
D Nagesh Kumar, IISc

4. Reservoir Operation – Steady State Policy…
Let there be T periods in a year, then the objective function is to maximize the total net benefits from all periods
Maximize
The constraints are
Continuity Equation after neglecting all the minor losses and assuming no overflow
Capacity constraint
Water Resources Planning and Management: M4L5
D Nagesh Kumar, IISc

5. Steady State Policy : Backward Recursion
Take stages as the time periods and the states as the storage volumes
Assume that there are T periods in a year
To find the steady state policy, select a period in a particular year, after which it is assumed that the reservoir is no longer useful (maximum lifetime of the reservoir)
Usually in almost all problems, the last period T is taken as the terminal period
At this stage, the optimal release will be independent of the inflow and also the net benefit.
Water Resources Planning and Management: M4L5
D Nagesh Kumar, IISc

6. Steady State Policy : Backward Recursion…
Consider the terminal period as T of a particular year after which reservoir is no longer useful
Solving this problem in a backward recursion method, let t represents the period in a year from T to 1 and n represents the periods remaining from t till end
t will take values starting from T, decreasing to 1 (which will complete one year) and then again taking a value of T and repeating the values
Value of n starts from 1 (while considering the Tth period of last year) and while moving backwards its value keeps on increasing i.e. at the beginning of the last year, the value of n = T and at the beginning of second last year its value will be equal to 2T and so on.
Water Resources Planning and Management: M4L5
D Nagesh Kumar, IISc

7. Steady State Policy : Backward Recursion…
Starting from the last period T of a particular year, there is only one period remaining
The maximum net benefit for this last period is
This is solved for all ST values from 0 to K
Water Resources Planning and Management: M4L5
D Nagesh Kumar, IISc

8. Steady State Policy : Backward Recursion…
Considering the last two stages together, the objective function is
ST-1 also ranges from 0 to K
In general, for a period t with n periods remaining, the function can be written as
where
the index t decreases from T to 1 and then takes the value T again and the cycle repeats for another year
the index n starts from 1 and increases at each successive stage
Water Resources Planning and Management: M4L5
D Nagesh Kumar, IISc

9. Steady State Policy : Backward Recursion…
The cycle is repeated till the optimum values of rt for an initial storage St will be the same as the corresponding rt and St of previous year
Such a solution is called stationary solution
Maximum net benefit is the difference of and for any St and t.
Water Resources Planning and Management: M4L5
D Nagesh Kumar, IISc

10. Steady State Policy : Numerical Example
Consider a reservoir for which the desirable constant storage is 20 units and the constant release is 25 units.
Capacity of the reservoir is 30 units.
Inflows for three seasons are 10, 50 and 20 units.
Optimization problem: To find the optimum rt and St that minimizes the total squared deviation from the release and storage targets given.
Thus, the objective function is
Minimize
Water Resources Planning and Management: M4L5
D Nagesh Kumar, IISc

11. Steady State Policy : Numerical Example…
St take the discrete values of 0, 10, 20, 30 and rt take the values of 10, 20, 30, 40
Solution:
Backward recursion is used to solve the problem
Consider a year after which the reservoir is no longer useful.
Expressing the problem as a sequential process as shown in the figure below
Water Resources Planning and Management: M4L5
D Nagesh Kumar, IISc

12. Steady State Policy : Numerical Example…
Considering the last period for which t = 3 and n = 1, the optimization function is
Minimize
Inflow for 3rd season, I3 = 20 units and capacity of the reservoir, K = 30 units
The release constraints can be expressed as
and
Water Resources Planning and Management: M4L5
D Nagesh Kumar, IISc

13. Steady State Policy : Numerical Example…
Computation for the first subproblem (n = 1)
Water Resources Planning and Management: M4L5
D Nagesh Kumar, IISc

14. Steady State Policy : Numerical Example…
Considering the last two periods (n =2), the optimization function is
Inflow for 2nd season, I2 = 50 units
Release constraints can be expressed as
and
While computing the optimal releases for S2=30,
Since r2 can take values only of 10, 20, 30 and 40 only, the release cannot be made for S2=30.
Water Resources Planning and Management: M4L5
D Nagesh Kumar, IISc

15. Steady State Policy : Numerical Example…
Computation for the second subproblem (n = 2)
Water Resources Planning and Management: M4L5
D Nagesh Kumar, IISc

16. Steady State Policy : Numerical Example…
The same procedure is repeated for all stages till n = 7. The summarized solution for this problem is given in the table below
Solution for the last year
Water Resources Planning and Management: M4L5
D Nagesh Kumar, IISc

17. Steady State Policy : Numerical Example…
Solution for the second last year
Water Resources Planning and Management: M4L5
D Nagesh Kumar, IISc

18. Steady State Policy : Numerical Example…
Solution for the last period of third last year
Water Resources Planning and Management: M4L5
D Nagesh Kumar, IISc

19. Steady State Policy : Numerical Example…
At this stage, the value of r3* at n = 7 and n = 4 are exactly the same. Also the difference is same for all St
This value is the minimum total squared deviations from the target release and storage
Stationary optimal policy
obtained is shown
Water Resources Planning and Management: M4L5
D Nagesh Kumar, IISc

20. Steady State Policy: Limitations
Assumption in dynamic programming: Decisions made at one stage is dependent only on the state variable and is independent of the decisions taken in other stages
Therefore dynamic programming will not be an appropriate technique where decisions are dependent on the earlier decisions.
Water Resources Planning and Management: M4L5
D Nagesh Kumar, IISc

21. Bibliography / Further Reading
Bellman, R., Dynamic Programming, Princeton University Press, Princeton, N.J, 1957.
Hillier F.S. and G.J. Lieberman, Operations Research, CBS Publishers & Distributors, New Delhi, 1987.
Loucks, D.P., J.R. Stedinger, and D.A. Haith, Water Resources Systems Planning and Analysis, Prentice-Hall, N.J., 1981.
Rao S.S., Engineering Optimization – Theory and Practice, Fourth Edition, John Wiley and Sons, 2009.
Taha H.A., Operations Research – An Introduction, 8th edition, Pearson Education India, 2008.
Vedula S., and P.P. Mujumdar, Water Resources Systems: Modelling Techniques and Analysis, Tata McGraw Hill, New Delhi, 2005.
Water Resources Planning and Management: M4L5
D Nagesh Kumar, IISc

22. Thank You Water Resources Planning and Management: M4L5
D Nagesh Kumar, IISc