Basic Concepts, Necessary and Sufficient conditions Dynamic Optimization Basic Concepts, Necessary and Sufficient conditions Lecture 30

Concept of function and functional … Functional: It is a function of other functions Examples: 𝑓( 𝑓 1 𝑥 , 𝑓 2 𝑥 … 𝑓 𝑛 𝑥 ) 𝑓(𝑥 𝑡 , 𝑥 𝑡 ,𝑡) 𝐽 𝑣 𝑡 = 𝑡 0 𝑡 𝑓 𝑣 𝑡 𝑑𝑡 It represents distance traveled, v(t) is velocity In general 𝐽 𝑥 𝑡 ,𝑡 = 𝑡 0 𝑡 𝑓 𝑣(𝑥 𝑡 ,𝑡)𝑑𝑡

Concept of function and functional … Increment of functional: Consider following functional 𝐽 𝑥 𝑡 ,𝑡 = 𝑡 0 𝑡 𝑓 𝑣(𝑥 𝑡 ,𝑡)𝑑𝑡 ∆𝐽 𝑥 𝑡 ,𝑡 =𝐽 𝑥 𝑡 +𝛿𝑥(𝑡),𝑡 −𝐽 𝑥 𝑡 ,𝑡 ---(1) Do Taylor’s series expansion of 2nd term ∆𝐽 𝑥 𝑡 ,𝑡 =𝐽 𝑥 ∗ 𝑡 ,𝑡 + 1 1! 𝜕𝐽 . 𝜕𝑥 𝑥 𝑡 = 𝑥 ∗ 𝑡 𝛿𝑥 𝑡 + 1 2! 𝜕 2 𝐽 . 𝜕𝑥 2 𝑥 𝑡 = 𝑥 ∗ 𝑡 𝛿𝑥(𝑡) 2 +…−𝐽 𝑥 ∗ 𝑡 ,𝑡 Perturbation

Concept of function and functional … ∆𝐽 𝑥 𝑡 ,𝑡 = 1 1! 𝜕𝐽 . 𝜕𝑥 𝑥 𝑡 = 𝑥 ∗ 𝑡 𝛿𝑥 𝑡 + 1 2! 𝜕 2 𝐽 . 𝜕𝑥 2 𝑥 𝑡 = 𝑥 ∗ 𝑡 𝛿𝑥(𝑡) 2 ∆𝐽=𝛿𝐽 +𝛿 2 𝐽 Necessary condition for a functional to be extremum (min or max) 𝛿𝐽=0 and If 𝛿 2 𝐽>0 functional value is minimum If 𝛿 2 𝐽<0 functional value is maximum 𝛿𝐽 1st variation of functional 𝛿 2 𝐽 2nd variation of functional Neglecting higher order terms

Two point boundary value problem (TPBVP)

is optimized (Extrimum -> maximize or minimize) Assumptions: Problem 1: Our problem is to find out the optimal trajectory x*(t) for which the functional 𝐽 𝑥 𝑡 ,𝑡 = 𝑡 0 𝑡 𝑓 𝑣(𝑥 𝑡 , 𝑥 𝑡 ,𝑡)𝑑𝑡 ---(1) is optimized (Extrimum -> maximize or minimize) Assumptions: V(.) has 1st and 2nd partial derivatives w.r.t all its arguments

Necessary Conditions: 𝜕𝑉 . 𝜕𝑥 𝑡 − 𝑑 𝑑𝑡 𝜕𝑉 . 𝜕 𝑥 𝑡 = 0 𝑛×1 ---(1) 𝑣 . − 𝜕𝑉 . 𝜕 𝑥 𝑡 𝑇 𝑥 (𝑡) 𝑡= 𝑡 𝑓 =0 ---(2) if 𝑡 𝑓 is free, ⇒𝛿 𝑡 𝑓 ≠0 𝜕𝑉 . 𝜕 𝑥 𝑡 𝑡= 𝑡 𝑓 =0 ---(3) if 𝑥(𝑡 𝑓 ) is free, ⇒𝛿 𝑥 𝑓 ≠0 (1) Euler’s Lagrange equation(E.L equation). (2) & (3) Transversality conditions

Necessary Conditions:… Case 1: when both end are fixed i.e 𝑡 𝑓 and 𝑥(𝑡 𝑓 ) are fixed We need to solve equation (1) to get optimum trajectory 𝑥 ∗ (𝑡)

Necessary Conditions:… A(t0, x(t0)) B(tf , x(tf)) t0 tf x(t0)=x0 x(tf)=xf t Say this is optimal path 𝑥 𝑎 𝑡 = 𝑥 ∗ 𝑡 +𝛿𝑥(𝑡) 𝑥 ∗ 𝑡 𝑥(𝑡) This is suboptimal path very close to x*(t) 𝛿𝑥(𝑡)

Necessary Conditions:… Case 2: when 𝑡 𝑓 and 𝑥(𝑡 𝑓 ) are free We need to solve equations (1), (2) & (3) to get optimum trajectory 𝑥 ∗ (𝑡) A t0 tf t 𝑥(𝑡) 𝑡 𝑓 +𝛿 𝑡 𝑓 D C 𝑥 ∗ (𝑡) 𝑥 𝑎 𝑡 = 𝑥 ∗ 𝑡 +𝛿𝑥(𝑡)

Necessary Conditions:… Case 3: when 𝑡 𝑓 is fixed but 𝑥(𝑡 𝑓 ) is free We need to solve equations (1) & (2) to get optimum trajectory 𝑥 ∗ (𝑡) t0 tf t 𝑥(𝑡) 𝑡 𝑓 +𝛿 𝑡 𝑓 𝑥 ∗ (𝑡) 𝑥 𝑎 𝑡 = 𝑥 ∗ 𝑡 +𝛿𝑥(𝑡) 𝑥(𝑡 𝑓 )

Necessary Conditions:… Case 4: when 𝑡 𝑓 is free but 𝑥(𝑡 𝑓 ) is fixed We need to solve equations (1) & (3) to get optimum trajectory 𝑥 ∗ (𝑡) t0 t 𝑥(𝑡) 𝑡 𝑓 𝑥 ∗ (𝑡) 𝑥 𝑎 𝑡 = 𝑥 ∗ 𝑡 +𝛿𝑥(𝑡)

Necessary Conditions:… Case 5: When end point B restricted to a curve g(t) t0 t 𝑥(𝑡) 𝑡 𝑓 𝑥 ∗ (𝑡) 𝑥 𝑎 𝑡 = 𝑥 ∗ 𝑡 +𝛿𝑥(𝑡) g(t) D C 𝛿𝑥 𝑓 =𝛿𝑥( 𝑡 𝑓 +𝛿 𝑡 𝑓 ) B A 𝑡 𝑓 +𝛿 𝑡 𝑓

Necessary Conditions:… 𝑣 . − 𝜕𝑉 . 𝜕 𝑥 𝑡 𝑇 𝑥 𝑡 − 𝑔 (𝑡) 𝑡= 𝑡 𝑓 =0 ---(4) Equ (4) is also known as Transversality conditions for the case when the end point B is restricted on a curve g(t). We need to solve equations (1) & (4) to get optimum trajectory 𝑥 ∗ (𝑡)

Sufficient condition Condition to check whether the given functional is maximum or minimum Similar to sufficient condition of static optimization problem

Sufficient condition… ⇒ 𝛻 2 𝐽= 1 2! 𝑡 0 𝑡 𝑓 𝛿𝑥(𝑡) 𝛿 𝑥 (𝑡) 𝜕 2 𝑉 . 𝜕 𝑥 2 𝑡 𝜕𝑉 . 𝜕𝑥 𝑡 𝜕 𝑥 𝑡 𝜕𝑉 . 𝜕𝑥 𝑡 𝜕 𝑥 𝑡 𝜕 2 𝑉 . 𝜕 𝑥 2 𝑡 ∗ 𝛿𝑥(𝑡) 𝛿 𝑥 (𝑡) 𝑑𝑡 Say P which is a 2nx2n matrix as each 2nd derivative of V(.) is nxn matrix

Sufficient condition… If 𝛻 2 𝐽>0⇒𝑃>0 (+𝑑𝑒𝑓𝑖𝑛𝑒𝑖𝑡𝑒) => functional value will be minimum =>J*(.) will be the minimum value along with the optimum trajectory x*(t) If 𝛻 2 𝐽<0⇒𝑃<0 (−𝑑𝑒𝑓𝑖𝑛𝑒𝑖𝑡𝑒) => functional value will be maximum =>J*(.) will be the maximum value along with the optimum trajectory x*(t)

Example: (Using calculus of variation) Example 1: Find the extremal for the functional 𝐽= 𝑡 0 =0 𝑡 𝑓 2 𝑥 2 𝑡 +24𝑡𝑥 𝑡 𝑑𝑡 Where the left hand point (A) is fixed i.e. t0 =0 & x(t0) =0 And in right hand point (B) tf is free but x(tf) = 2

Example:… Solution: This is following problem 𝑉 . =2 𝑥 2 𝑡 +24𝑡𝑥 𝑡 𝑉 . =2 𝑥 2 𝑡 +24𝑡𝑥 𝑡 t0 tf t 𝑥(𝑡) 𝑡 𝑓 +𝛿 𝑡 𝑓 𝑥 ∗ (𝑡) 𝑥 𝑎 𝑡 = 𝑥 ∗ 𝑡 +𝛿𝑥(𝑡) 𝑥(𝑡 𝑓 )=2

Example:… E.L equation 𝜕𝑉 . 𝜕𝑥 𝑡 − 𝑑 𝑑𝑡 𝜕𝑉 . 𝜕 𝑥 𝑡 = 0 1×1 (x is scalar variable) ⇒24𝑡− 𝑑 𝑑𝑡 4 𝑥 (𝑡) =0 ⇒24𝑡−4 𝑥 (𝑡)=0 ⇒ 𝑥 (𝑡)=6𝑡 ---(1) Solving above equation, Integrating equ(1) 𝑥 𝑡 𝑑𝑡= 6𝑡𝑑𝑡 ⇒ 𝑥 𝑡 =3 𝑡 2 + 𝑐 1 ---(2) again integrating ⇒ 𝑥 𝑡 𝑑𝑡= 3 𝑡 2 + 𝑐 1 𝑑𝑡 ⇒𝑥 𝑡 = 𝑡 3 + 𝑐 1 𝑡+ 𝑐 2 ---(3)

Example:… Put t=t0=0 in equation (3) 𝑥 𝑡 =0= 0 3 + 𝑐 1 ×0+ 𝑐 2 ⇒𝑐 2 =0 put in equation (3) ⇒𝑥 𝑡 = 𝑡 3 + 𝑐 1 𝑡 ---(4) Put t=tf & 𝑥 𝑡 𝑓 =2 in equation (4) 𝑥 𝑡 𝑓 =2= 𝑡 𝑓 3 + 𝑐 1 𝑡 𝑓 ⇒𝑐 1 = 2− 𝑡 𝑓 3 𝑡 𝑓 ---(5)

Example:… Transversality condition: Here 𝑡 𝑓 is free, so 𝑣 . − 𝜕𝑉 . 𝜕 𝑥 𝑡 𝑇 𝑥 (𝑡) 𝑡= 𝑡 𝑓 =0 2 𝑥 2 𝑡 +24𝑡𝑥 𝑡 −(4 𝑥 (𝑡)) 𝑥 (𝑡) 𝑡= 𝑡 𝑓 =0 dividing by 2 ⇒ 𝑥 2 𝑡 𝑓 +12 𝑡 𝑓 𝑥 𝑡 𝑓 −2 𝑥 2 ( 𝑡 𝑓 ) ⇒ 𝑥 2 𝑡 𝑓 −12 𝑡 𝑓 𝑥 𝑡 𝑓 =0 put value of 𝑥 (𝑡) from equ(2) ⇒ 3 𝑡 𝑓 2 + 𝑐 1 2 −12 𝑡 𝑓 𝑥 𝑡 𝑓 =0 put 𝑥 𝑡 𝑓 =2 ⇒9 𝑡 𝑓 4 + 𝑐 1 2 +6 𝑡 𝑓 2 𝑐 1 −24 𝑡 𝑓 =0

Example:… Put the value of 𝑐 1 = 2− 𝑡 𝑓 3 𝑡 𝑓 from equ (5) in previous equ ⇒9 𝑡 𝑓 4 + 2− 𝑡 𝑓 3 𝑡 𝑓 2 +6 𝑡 𝑓 2 2− 𝑡 𝑓 3 𝑡 𝑓 −24 𝑡 𝑓 =0 ⇒𝑡 𝑓 6 −4 𝑡 𝑓 3 +1=0 Put 𝑡 𝑓 3 =𝑧 𝑧 2 −4𝑧+1=0 ⇒𝑧=2± 3 =3.732, 0.2679 ⇒ 𝑡 𝑓 = 3 3.732 & 3 0.2679 ⇒ 𝑡 𝑓 =1.551 & 0.6446

Example:… For 𝑡 𝑓 =1.551 , 𝑐 1 = 2− 𝑡 𝑓 3 𝑡 𝑓 =−1.117 so 𝑥 ∗ 𝑡 = 𝑡 3 + 𝑐 1 𝑡= 𝑡 3 −1.117𝑡 ---(6) For 𝑡 𝑓 =0.6446 , 𝑐 1 = 2− 𝑡 𝑓 3 𝑡 𝑓 =2.6871so 𝑥 ∗ 𝑡 = 𝑡 3 + 𝑐 1 𝑡= 𝑡 3 +2.6871𝑡 --(7) There are two optimal trajectory given by (6) & (7), one will give minimum value other will give maximum value. For this apply sufficient condition

Example:… sufficient condition 𝑃= 𝜕 2 𝑉 . 𝜕 𝑥 2 𝑡 𝜕𝑉 . 𝜕𝑥 𝑡 𝜕 𝑥 𝑡 𝜕𝑉 . 𝜕𝑥 𝑡 𝜕 𝑥 𝑡 𝜕 2 𝑉 . 𝜕 𝑥 2 𝑡 ∗ If 𝑃>0 (+𝑑𝑒𝑓𝑖𝑛𝑒𝑖𝑡𝑒) => functional J*(.) value will be minimum If 𝑃<0 (−𝑑𝑒𝑓𝑖𝑛𝑒𝑖𝑡𝑒) => functional J*(.) value will be maximum