Vector Calculus and Quadratic function 10/13/2018 By Nafees Ahamad

Gradient of a function Consider a multivariable, scalar function f(x) f(x)=f(xnx1)=f(x1,x2…xn) where 𝑥=𝑥 𝑛×1 = 𝑥 1 𝑥 2 . . . 𝑥 𝑛 𝑖𝑠 𝑎 𝑣𝑒𝑐𝑡𝑜𝑟 Gradient of f(x) 𝛻𝑓 𝑥 = 𝑔𝑟𝑎𝑑 𝑓(𝑥) 𝑥=𝑥∗ = 𝜕𝑓(𝑥) 𝜕𝑥 𝑥=𝑥∗ where 𝑥=𝑥 𝑛×1 =𝑥 𝑛×1 ∗ = 𝑥 1 ∗ 𝑥 2 ∗ . . . 𝑥 𝑛 ∗ 𝑖𝑠 𝑎 𝑣𝑒𝑐𝑡𝑜𝑟 10/13/2018 By Nafees Ahamad

Gradient of a function… 𝛻𝑓 𝑥 = 𝜕𝑓(𝑥) 𝜕 𝑥 1 𝜕𝑓(𝑥) 𝜕 𝑥 2 . . . 𝜕𝑓(𝑥) 𝜕 𝑥 𝑛 𝑥=𝑥∗ = 𝜕𝑓(𝑥) 𝜕 𝑥 1 𝜕𝑓(𝑥) 𝜕 𝑥 2 . . . 𝜕𝑓(𝑥) 𝜕 𝑥 𝑛 𝑥=𝑥∗ 𝑇 So gradient of a scalar function is a column vector (Gradient=> derivative of a scalar function w.r.t vector) T→Transpose 10/13/2018 By Nafees Ahamad

Example 1: Calculate the gradient vector for the function 𝑓 𝑥 = 𝑥 1 2 +2 𝑥 1 𝑥 2 + 3𝑥 2 2 +4 𝑥 1 𝑥 2 𝑥 3 + 𝑥 3 3 at 𝑥 3×1 = 1 2 0 𝑇 Solution 𝜕𝑓(𝑥) 𝜕 𝑥 1 =2 𝑥 1 +2 𝑥 2 +4 𝑥 2 𝑥 3 𝜕𝑓(𝑥) 𝜕 𝑥 2 =2 𝑥 1 +6 𝑥 2 +4 𝑥 1 𝑥 3 𝜕𝑓(𝑥) 𝜕 𝑥 3 =4 𝑥 1 𝑥 2 +3 𝑥 3 2 10/13/2018 By Nafees Ahamad

Example 1… So gradient 𝛻𝑓 𝑥 = 𝜕𝑓(𝑥) 𝜕 𝑥 1 𝜕𝑓(𝑥) 𝜕 𝑥 2 𝜕𝑓(𝑥) 𝜕 𝑥 3 𝑥=𝑥∗ = 2 𝑥 1 +2 𝑥 2 +4 𝑥 2 𝑥 3 2 𝑥 1 +6 𝑥 2 +4 𝑥 1 𝑥 3 4 𝑥 1 𝑥 2 +3 𝑥 3 2 𝑥= 𝑥 ∗ = 1 2 0 = 6 14 8 Geometrically , the gradient of a function is normal to the tangent plane at the point x=x* 10/13/2018 By Nafees Ahamad

2nd derivative of a real valued function: First take the derivative of a function (scalar) that means gradient of a function. Which results into a vector. Now again take the derivate of this vector w.r.t vector which will result into a matrix. 𝛻 2 𝑓 𝑥 = 𝜕 𝜕𝑓(𝑥) 𝜕𝑥 𝜕𝑥 𝑥=𝑥∗ = 𝜕 2 𝑓(𝑥) 𝜕𝑥 2 𝑥=𝑥∗ 𝛻 2 𝑓 𝑥 = 𝜕 𝜕𝑥 𝜕𝑓(𝑥) 𝜕 𝑥 1 𝜕𝑓(𝑥) 𝜕 𝑥 2 . . . 𝜕𝑓(𝑥) 𝜕 𝑥 𝑛 𝑥=𝑥∗ 𝑇 10/13/2018 By Nafees Ahamad

2nd derivative of a real valued function… 𝛻 2 𝑓 𝑥 = 𝜕 2 𝑓(𝑥) 𝜕 𝑥 1 2 𝜕 2 𝑓(𝑥) 𝜕 𝑥 1 𝑥 2 . . . 𝜕𝑓(𝑥) 𝜕 𝑥 1 𝑥 𝑛 𝜕 2 𝑓(𝑥) 𝜕 𝑥 1 𝑥 2 𝜕 2 𝑓(𝑥) 𝜕 𝑥 2 2 . . . 𝜕𝑓(𝑥) 𝜕 𝑥 2 𝑥 𝑛 . . . 𝜕 2 𝑓(𝑥) 𝜕 𝑥 1 𝑥 𝑛 𝜕 2 𝑓(𝑥) 𝜕 𝑥 2 𝑥 𝑛 . . . 𝜕 2 𝑓(𝑥) 𝜕 𝑥 𝑛 2 𝑥= 𝑥 ∗ It is a symmetrical nxn matrix (aij=aji i≠j ) and known Hessian Matrix 10/13/2018 By Nafees Ahamad

Example 2 Calculate 2nd derivative of function 𝑓 𝑥 = 𝑥 1 2 +2 𝑥 1 𝑥 2 + 3𝑥 2 2 +4 𝑥 1 𝑥 2 𝑥 3 + 𝑥 3 3 at 𝑥 3×1 = 1 2 0 𝑇 Solution: 𝜕𝑓(𝑥) 𝜕 𝑥 1 =2 𝑥 1 +2 𝑥 2 +4 𝑥 2 𝑥 3 𝜕𝑓(𝑥) 𝜕 𝑥 2 =2 𝑥 1 +6 𝑥 2 +4 𝑥 1 𝑥 3 𝜕𝑓(𝑥) 𝜕 𝑥 3 =4 𝑥 1 𝑥 2 +3 𝑥 3 2 10/13/2018 By Nafees Ahamad

Example 2… 𝛻 2 𝑓 𝑥 = 𝜕 2 𝑓(𝑥) 𝜕 𝑥 1 2 𝜕 2 𝑓(𝑥) 𝜕 𝑥 1 𝑥 2 𝜕 2 𝑓(𝑥) 𝜕 𝑥 1 𝑥 3 𝜕 2 𝑓(𝑥) 𝜕 𝑥 1 𝑥 2 𝜕 2 𝑓(𝑥) 𝜕 𝑥 2 2 𝜕 2 𝑓(𝑥) 𝜕 𝑥 2 𝑥 3 𝜕 2 𝑓(𝑥) 𝜕 𝑥 1 𝑥 3 𝜕 2 𝑓(𝑥) 𝜕 𝑥 2 𝑥 3 𝜕 2 𝑓(𝑥) 𝜕 𝑥 3 2 𝑥= 𝑥 ∗ 10/13/2018 By Nafees Ahamad

Example 2… 𝜕 2 𝑓(𝑥) 𝜕 𝑥 1 2 =2 𝜕 2 𝑓(𝑥) 𝜕 𝑥 1 𝑥 2 =2+4 𝑥 3 𝜕 2 𝑓(𝑥) 𝜕 𝑥 1 2 =2 𝜕 2 𝑓(𝑥) 𝜕 𝑥 1 𝑥 2 =2+4 𝑥 3 𝜕 2 𝑓(𝑥) 𝜕 𝑥 1 𝑥 3 =4 𝑥 2 𝜕 2 𝑓(𝑥) 𝜕 𝑥 2 2 =6 𝜕 2 𝑓(𝑥) 𝜕 𝑥 2 𝑥 3 =4 𝑥 1 𝜕 2 𝑓(𝑥) 𝜕 𝑥 3 2 =6 𝑥 3 𝛻 2 𝑓 𝑥 = 2 2+4 𝑥 3 4 𝑥 2 2+4 𝑥 3 6 4 𝑥 1 4 𝑥 2 4 𝑥 1 6 𝑥 3 𝑥= 𝑥 ∗ = 1 2 0 𝑇 𝛻 2 𝑓 𝑥 = 2 2 8 2 6 4 8 4 0 10/13/2018 By Nafees Ahamad

Quadratic functions Suppose we have a function of two variables x1 & x2 The quadratic function for these variables may be 𝑓 𝑥 1 , 𝑥 2 =𝑎 𝑥 1 2 +𝑏 𝑥 1 𝑥 2 +𝑐 𝑥 2 2 −−−(1) Above equation (1) may be written as 𝑓 𝑥 1 , 𝑥 2 = 𝑥 1 𝑥 2 𝑎 𝑏 0 𝑐 𝑥 1 𝑥 2 = 𝑥 𝑇 𝑃𝑥 Or 𝑓 𝑥 1 , 𝑥 2 = 𝑥 1 𝑥 2 𝑎 0 𝑏 𝑐 𝑥 1 𝑥 2 = 𝑥 𝑇 𝑃𝑥 We can change above diagonal elements and get many (infinite) solution Say Matrix P 10/13/2018 By Nafees Ahamad

Quadratic functions … 𝑓 𝑥 1 , 𝑥 2 = 𝑥 1 𝑥 2 𝑎 −𝑏 2𝑏 𝑐 𝑥 1 𝑥 2 = 𝑥 𝑇 𝑃𝑥 𝑓 𝑥 1 , 𝑥 2 = 𝑥 1 𝑥 2 𝑎 −𝑏 2𝑏 𝑐 𝑥 1 𝑥 2 = 𝑥 𝑇 𝑃𝑥 𝑓 𝑥 1 , 𝑥 2 = 𝑥 1 𝑥 2 𝑎 𝑏 2 𝑏 2 𝑐 𝑥 1 𝑥 2 = 𝑥 𝑇 𝑃𝑥 So Matrix P is not unique P=Symmetrical matrix 10/13/2018 By Nafees Ahamad

General Quadratic function A quadratic function of ‘n’ variables 𝑓 𝑥 1 , 𝑥 2 , 𝑥 3 ,… 𝑥 𝑛 =𝑓 𝑥 𝑛×1 = 𝑥 𝑇 𝑃𝑥 Where 𝑥= 𝑥 1 𝑥 2 . . . 𝑥 𝑛 𝑛×1 and Pnxn symmetric matrix 10/13/2018 By Nafees Ahamad

General Quadratic function… Suppose P is not a symmetric matrix then 𝑓 𝑥 = 𝑥 𝑇 𝑃𝑥= 1 2 𝑥 𝑇 𝑃𝑥+ 1 2 𝑥 𝑇 𝑃𝑥 𝑓 𝑥 = 𝑥 𝑇 𝑃𝑥= 1 2 𝑥 𝑇 𝑃𝑥+ 1 2 𝑥 𝑇 𝑃𝑥 𝑇 𝑓 𝑥 = 𝑥 𝑇 𝑃𝑥= 1 2 𝑥 𝑇 𝑃𝑥+ 1 2 𝑥 𝑇 𝑃 𝑇 𝑥 ∵ 𝐴𝐵𝐶 𝑇 = 𝐶 𝑇 𝐵 𝑇 𝐴 𝑇 & 𝑥 𝑇 𝑇 =𝑥 𝑓 𝑥 = 𝑥 𝑇 𝑃𝑥= 𝑥 𝑇 𝑃+ 𝑃 𝑇 2 𝑥= 𝑥 𝑇 Qx f(x) will be scalar function i.e f(x)=10 say, so it may be written as F(x)=5+5 1 2 𝑥 𝑇 𝑃𝑥 is scalar so we can take transpose of it Always Symmetrical matrix 10/13/2018 By Nafees Ahamad

Next: Now some other concepts like + definite, + semi definite, - definite & - sem definite. 10/13/2018 By Nafees Ahamad