1. Basic Concepts, Necessary and Sufficient conditions
Dynamic Optimization
Basic Concepts, Necessary and Sufficient conditions
Lecture 30

2. Concept of function and functional …
Functional: It is a function of other functions
Examples:
𝑓( 𝑓 1 𝑥 , 𝑓 2 𝑥 … 𝑓 𝑛 𝑥 )
𝑓(𝑥 𝑡 , 𝑥 𝑡 ,𝑡)
𝐽 𝑣 𝑡 = 𝑡 0 𝑡 𝑓 𝑣 𝑡 𝑑𝑡 It represents distance traveled, v(t) is velocity
In general
𝐽 𝑥 𝑡 ,𝑡 = 𝑡 0 𝑡 𝑓 𝑣(𝑥 𝑡 ,𝑡)𝑑𝑡

3. Concept of function and functional …
Increment of functional:
Consider following functional 𝐽 𝑥 𝑡 ,𝑡 = 𝑡 0 𝑡 𝑓 𝑣(𝑥 𝑡 ,𝑡)𝑑𝑡
∆𝐽 𝑥 𝑡 ,𝑡 =𝐽 𝑥 𝑡 +𝛿𝑥(𝑡),𝑡 −𝐽 𝑥 𝑡 ,𝑡 (1)
Do Taylor’s series expansion of 2nd term
∆𝐽 𝑥 𝑡 ,𝑡 =𝐽 𝑥 ∗ 𝑡 ,𝑡 + 1 1! 𝜕𝐽 . 𝜕𝑥 𝑥 𝑡 = 𝑥 ∗ 𝑡 𝛿𝑥 𝑡 ! 𝜕 2 𝐽 . 𝜕𝑥 2 𝑥 𝑡 = 𝑥 ∗ 𝑡 𝛿𝑥(𝑡) 2 +…−𝐽 𝑥 ∗ 𝑡 ,𝑡
Perturbation

4. Concept of function and functional …
∆𝐽 𝑥 𝑡 ,𝑡 = 1 1! 𝜕𝐽 . 𝜕𝑥 𝑥 𝑡 = 𝑥 ∗ 𝑡 𝛿𝑥 𝑡 + 1 2! 𝜕 2 𝐽 . 𝜕𝑥 2 𝑥 𝑡 = 𝑥 ∗ 𝑡 𝛿𝑥(𝑡) 2
∆𝐽=𝛿𝐽 +𝛿 2 𝐽
Necessary condition for a functional to be extremum (min or max)
𝛿𝐽=0 and
If 𝛿 2 𝐽>0 functional value is minimum
If 𝛿 2 𝐽<0 functional value is maximum 𝛿𝐽
1st variation of functional
𝛿 2 𝐽
2nd variation of functional
Neglecting higher order terms

5. Two point boundary value problem (TPBVP)

6. is optimized (Extrimum -> maximize or minimize) Assumptions:
Problem 1: Our problem is to find out the optimal trajectory x*(t) for which the functional 𝐽 𝑥 𝑡 ,𝑡 = 𝑡 0 𝑡 𝑓 𝑣(𝑥 𝑡 , 𝑥 𝑡 ,𝑡)𝑑𝑡 ---(1)
is optimized (Extrimum -> maximize or minimize)
Assumptions:
V(.) has 1st and 2nd partial derivatives w.r.t all its arguments

7. Necessary Conditions:
𝜕𝑉 . 𝜕𝑥 𝑡 − 𝑑 𝑑𝑡 𝜕𝑉 . 𝜕 𝑥 𝑡 = 0 𝑛× (1)
𝑣 . − 𝜕𝑉 . 𝜕 𝑥 𝑡 𝑇 𝑥 (𝑡) 𝑡= 𝑡 𝑓 =0 ---(2) if 𝑡 𝑓 is free, ⇒𝛿 𝑡 𝑓 ≠0
𝜕𝑉 . 𝜕 𝑥 𝑡 𝑡= 𝑡 𝑓 = (3) if 𝑥(𝑡 𝑓 ) is free, ⇒𝛿 𝑥 𝑓 ≠0
(1) Euler’s Lagrange equation(E.L equation).
(2) & (3) Transversality conditions

8. Necessary Conditions:…
Case 1:
when both end are fixed i.e 𝑡 𝑓 and 𝑥(𝑡 𝑓 ) are fixed
We need to solve equation (1) to get optimum trajectory 𝑥 ∗ (𝑡)

9. Necessary Conditions:…
A(t0, x(t0))
B(tf , x(tf)) t0 tf
x(t0)=x0
x(tf)=xf t
Say this is optimal path
𝑥 𝑎 𝑡 = 𝑥 ∗ 𝑡 +𝛿𝑥(𝑡)
𝑥 ∗ 𝑡
𝑥(𝑡)
This is suboptimal path very close to x*(t)
𝛿𝑥(𝑡)

10. Necessary Conditions:…
Case 2:
when 𝑡 𝑓 and 𝑥(𝑡 𝑓 ) are free
We need to solve equations (1), (2) & (3) to get optimum trajectory 𝑥 ∗ (𝑡) A t0 tf t
𝑥(𝑡)
𝑡 𝑓 +𝛿 𝑡 𝑓 D C
𝑥 ∗ (𝑡)
𝑥 𝑎 𝑡 = 𝑥 ∗ 𝑡 +𝛿𝑥(𝑡)

11. Necessary Conditions:…
Case 3:
when 𝑡 𝑓 is fixed but 𝑥(𝑡 𝑓 ) is free
We need to solve equations (1) & (2) to get optimum trajectory 𝑥 ∗ (𝑡) t0 tf t
𝑥(𝑡)
𝑡 𝑓 +𝛿 𝑡 𝑓
𝑥 ∗ (𝑡)
𝑥 𝑎 𝑡 = 𝑥 ∗ 𝑡 +𝛿𝑥(𝑡)
𝑥(𝑡 𝑓 )

12. Necessary Conditions:…
Case 4:
when 𝑡 𝑓 is free but 𝑥(𝑡 𝑓 ) is fixed
We need to solve equations (1) & (3) to get optimum trajectory 𝑥 ∗ (𝑡) t0 t
𝑥(𝑡)
𝑡 𝑓
𝑥 ∗ (𝑡)
𝑥 𝑎 𝑡 = 𝑥 ∗ 𝑡 +𝛿𝑥(𝑡)

13. Necessary Conditions:…
Case 5:
When end point B restricted to a curve g(t) t0 t
𝑥(𝑡)
𝑡 𝑓
𝑥 ∗ (𝑡)
𝑥 𝑎 𝑡 = 𝑥 ∗ 𝑡 +𝛿𝑥(𝑡)
g(t) D C
𝛿𝑥 𝑓 =𝛿𝑥( 𝑡 𝑓 +𝛿 𝑡 𝑓 ) B A
𝑡 𝑓 +𝛿 𝑡 𝑓

14. Necessary Conditions:…
𝑣 . − 𝜕𝑉 . 𝜕 𝑥 𝑡 𝑇 𝑥 𝑡 − 𝑔 (𝑡) 𝑡= 𝑡 𝑓 =0 ---(4)
Equ (4) is also known as Transversality conditions for the case when the end point B is restricted on a curve g(t).
We need to solve equations (1) & (4) to get optimum trajectory 𝑥 ∗ (𝑡)

15. Sufficient condition
Condition to check whether the given functional is maximum or minimum
Similar to sufficient condition of static optimization problem

16. Sufficient condition…
⇒ 𝛻 2 𝐽= 1 2! 𝑡 0 𝑡 𝑓 𝛿𝑥(𝑡) 𝛿 𝑥 (𝑡) 𝜕 2 𝑉 . 𝜕 𝑥 2 𝑡 𝜕𝑉 . 𝜕𝑥 𝑡 𝜕 𝑥 𝑡 𝜕𝑉 . 𝜕𝑥 𝑡 𝜕 𝑥 𝑡 𝜕 2 𝑉 . 𝜕 𝑥 2 𝑡 ∗ 𝛿𝑥(𝑡) 𝛿 𝑥 (𝑡) 𝑑𝑡
Say P which is a 2nx2n matrix
as each 2nd derivative of V(.) is nxn matrix

17. Sufficient condition…
If 𝛻 2 𝐽>0⇒𝑃>0 (+𝑑𝑒𝑓𝑖𝑛𝑒𝑖𝑡𝑒)
=> functional value will be minimum
=>J*(.) will be the minimum value along with the optimum trajectory x*(t)
If 𝛻 2 𝐽<0⇒𝑃<0 (−𝑑𝑒𝑓𝑖𝑛𝑒𝑖𝑡𝑒)
=> functional value will be maximum
=>J*(.) will be the maximum value along with the optimum trajectory x*(t)

18. Example: (Using calculus of variation)
Example 1: Find the extremal for the functional
𝐽= 𝑡 0 =0 𝑡 𝑓 2 𝑥 2 𝑡 +24𝑡𝑥 𝑡 𝑑𝑡
Where the left hand point (A) is fixed i.e. t0 =0 & x(t0) =0
And in right hand point (B) tf is free but x(tf) = 2

19. Example:… Solution: This is following problem 𝑉 . =2 𝑥 2 𝑡 +24𝑡𝑥 𝑡
𝑉 . =2 𝑥 2 𝑡 +24𝑡𝑥 𝑡 t0 tf t
𝑥(𝑡)
𝑡 𝑓 +𝛿 𝑡 𝑓
𝑥 ∗ (𝑡)
𝑥 𝑎 𝑡 = 𝑥 ∗ 𝑡 +𝛿𝑥(𝑡)
𝑥(𝑡 𝑓 )=2

20. Example:…
E.L equation
𝜕𝑉 . 𝜕𝑥 𝑡 − 𝑑 𝑑𝑡 𝜕𝑉 . 𝜕 𝑥 𝑡 = 0 1×1 (x is scalar variable)
⇒24𝑡− 𝑑 𝑑𝑡 4 𝑥 (𝑡) =0
⇒24𝑡−4 𝑥 (𝑡)=0
⇒ 𝑥 (𝑡)=6𝑡 ---(1)
Solving above equation, Integrating equ(1)
𝑥 𝑡 𝑑𝑡= 6𝑡𝑑𝑡 ⇒ 𝑥 𝑡 =3 𝑡 2 + 𝑐 (2) again integrating
⇒ 𝑥 𝑡 𝑑𝑡= 3 𝑡 2 + 𝑐 1 𝑑𝑡
⇒𝑥 𝑡 = 𝑡 3 + 𝑐 1 𝑡+ 𝑐 (3)

21. Example:… Put t=t0=0 in equation (3) 𝑥 𝑡 =0= 0 3 + 𝑐 1 ×0+ 𝑐 2
⇒𝑐 2 =0 put in equation (3)
⇒𝑥 𝑡 = 𝑡 3 + 𝑐 1 𝑡 ---(4)
Put t=tf & 𝑥 𝑡 𝑓 =2 in equation (4)
𝑥 𝑡 𝑓 =2= 𝑡 𝑓 3 + 𝑐 1 𝑡 𝑓
⇒𝑐 1 = 2− 𝑡 𝑓 3 𝑡 𝑓 (5)

22. Example:… Transversality condition: Here 𝑡 𝑓 is free, so
𝑣 . − 𝜕𝑉 . 𝜕 𝑥 𝑡 𝑇 𝑥 (𝑡) 𝑡= 𝑡 𝑓 =0
2 𝑥 2 𝑡 +24𝑡𝑥 𝑡 −(4 𝑥 (𝑡)) 𝑥 (𝑡) 𝑡= 𝑡 𝑓 =0 dividing by 2
⇒ 𝑥 2 𝑡 𝑓 +12 𝑡 𝑓 𝑥 𝑡 𝑓 −2 𝑥 2 ( 𝑡 𝑓 )
⇒ 𝑥 2 𝑡 𝑓 −12 𝑡 𝑓 𝑥 𝑡 𝑓 =0 put value of 𝑥 (𝑡) from equ(2)
⇒ 3 𝑡 𝑓 2 + 𝑐 −12 𝑡 𝑓 𝑥 𝑡 𝑓 =0 put 𝑥 𝑡 𝑓 =2
⇒9 𝑡 𝑓 4 + 𝑐 𝑡 𝑓 2 𝑐 1 −24 𝑡 𝑓 =0

23. Example:…
Put the value of 𝑐 1 = 2− 𝑡 𝑓 3 𝑡 𝑓 from equ (5) in previous equ
⇒9 𝑡 𝑓 − 𝑡 𝑓 3 𝑡 𝑓 𝑡 𝑓 2 2− 𝑡 𝑓 3 𝑡 𝑓 −24 𝑡 𝑓 =0
⇒𝑡 𝑓 6 −4 𝑡 𝑓 3 +1=0
Put 𝑡 𝑓 3 =𝑧
𝑧 2 −4𝑧+1=0
⇒𝑧=2± 3 =3.732,
⇒ 𝑡 𝑓 = &
⇒ 𝑡 𝑓 =1.551 &

24. Example:… For 𝑡 𝑓 =1.551 , 𝑐 1 = 2− 𝑡 𝑓 3 𝑡 𝑓 =−1.117 so
𝑥 ∗ 𝑡 = 𝑡 3 + 𝑐 1 𝑡= 𝑡 3 −1.117𝑡 ---(6)
For 𝑡 𝑓 = , 𝑐 1 = 2− 𝑡 𝑓 3 𝑡 𝑓 =2.6871so
𝑥 ∗ 𝑡 = 𝑡 3 + 𝑐 1 𝑡= 𝑡 𝑡 --(7)
There are two optimal trajectory given by (6) & (7), one will give minimum value other will give maximum value. For this apply sufficient condition

25. Example:… sufficient condition
𝑃= 𝜕 2 𝑉 . 𝜕 𝑥 2 𝑡 𝜕𝑉 . 𝜕𝑥 𝑡 𝜕 𝑥 𝑡 𝜕𝑉 . 𝜕𝑥 𝑡 𝜕 𝑥 𝑡 𝜕 2 𝑉 . 𝜕 𝑥 2 𝑡 ∗
If 𝑃>0 (+𝑑𝑒𝑓𝑖𝑛𝑒𝑖𝑡𝑒) => functional J*(.) value will be minimum
If 𝑃<0 (−𝑑𝑒𝑓𝑖𝑛𝑒𝑖𝑡𝑒) => functional J*(.) value will be maximum