Dynamic Programming
A Simple Example
Capital as a State Variable
The Value Function
The Functional Equation
Computing the Value Function
The standard approach for computing value functions is known as “iteration on the value function” 1.To proceed, first specify functional forms and parameter values for the primitives of the problem: utility functions, production functions, the discount factor, etc. 2.Then set up a grid of possible values of the state variable: in cases where the state variable is continuous (such as the capital accumulation problem) this results in an approximate solution rather than an exact one, but this is usually good enough 3.The next step is to choose an initial value function; this sounds difficult but in practice it usually doesn’t matter too much what you start with: setting V(k) = 0 for every k on the grid is a typical choice
Iteration on the Value Function
Mathematical Foundations We would like our model to yield a unique solution for V(.) (a unique function, not a point) This means no matter which initial value function we choose at the start of the iterations, the final value function is always the same Similarly, we would like g(.) to be single-valued (a function instead of a correspondence) Then, given an initial state, the model would yield a unique prediction for the entire path of capital accumulation Whether the model yields unique solutions for the value function and policy function depends on the primitives of the problem: utility functions, production functions, etc. Stokey and Lucas describe general sufficient conditions for a model to yield a unique value function and a unique policy function (these conditions are sufficient, not necessary)
General Notation
Assumptions
Key Results
The Content of the Assumptions
The Assumptions