1. + Definite, + semi definite, - definite & - semi definite functions

2. +/- Definite quadratic function
Consider n variables quadratic function
𝑓 𝑥 𝑛×1 = 𝑝 11 𝑥 𝑝 12 𝑥 1 𝑥 2 + 𝑝 13 𝑥 1 𝑥 3 … 𝑝 1𝑛 𝑥 1 𝑥 𝑛 +
𝑝 21 𝑥 2 𝑥 1 + 𝑝 22 𝑥 𝑝 23 𝑥 2 𝑥 3 … 𝑝 2𝑛 𝑥 2 𝑥 𝑛 +
… 𝑝 𝑛1 𝑥 𝑛 𝑥 1 + 𝑝 𝑛2 𝑥 𝑛 𝑥 2 + 𝑝 𝑛3 𝑥 𝑛 𝑥 3 … 𝑝 𝑛𝑛 𝑥 𝑛 2 So
𝑓 𝑥 𝑛×1 = 𝑥 𝑇 𝑃𝑥 (p12≠p21 etc => Pnxn is non symmetrical matrix)

3. +/- Definite quadratic function …
Above quadratic function 𝑓 𝑥 = 𝑥 𝑇 𝑃𝑥 is said to be + definite if
𝑓 𝑥 = 𝑥 𝑇 𝑃𝑥> ∀ 𝑥≠ 0 𝑛×1
Above quadratic function 𝑓 𝑥 = 𝑥 𝑇 𝑃𝑥 is said to be + semi definite if
𝑓 𝑥 = 𝑥 𝑇 𝑃𝑥≥ ∀ 𝑥≠ 0 𝑛×1
Above quadratic function 𝑓 𝑥 = 𝑥 𝑇 𝑃𝑥 is said to be - definite if
𝑓 𝑥 = 𝑥 𝑇 𝑃𝑥< ∀ 𝑥≠ 0 𝑛×1
Above quadratic function 𝑓 𝑥 = 𝑥 𝑇 𝑃𝑥 is said to be – semi definite if
𝑓 𝑥 = 𝑥 𝑇 𝑃𝑥≤ ∀ 𝑥≠ 0 𝑛×1

4. Indefinite quadratic function
Above quadratic function 𝑓 𝑥 = 𝑥 𝑇 𝑃𝑥 is said to be indefinite if 𝑓 𝑥 = 𝑥 𝑇 𝑃𝑥>0 𝑓𝑜𝑟 𝑠𝑜𝑚𝑒 𝑜𝑓 𝑥≠ 0 𝑛×1 𝑓 𝑥 = 𝑥 𝑇 𝑃𝑥<0 𝑜𝑡ℎ𝑒𝑟 𝑣𝑎𝑙𝑢𝑒𝑠 𝑜𝑓 𝑥≠ 0 𝑛×1

5. +/- Definite matrix
A matrix Pnxn (symmetrical or non symmetrical) is + definite
iff 𝑥 𝑇 𝑃𝑥> ∀ 𝑥≠ 0 𝑛×1
In short
If P>0 => +definite matrix
If 𝑃= 𝑃 𝑇 >0 => Symmetrical + definite matrix
A matrix Pnxn (symmetrical or non symmetrical) is + semi definite
iff 𝑥 𝑇 𝑃𝑥≥ ∀ 𝑥≠ 0 𝑛×1
If P ≥ 0 => + semi definite matrix
If 𝑃= 𝑃 𝑇 ≥0 => Symmetrical + semi definite matrix
=> Matrix P is converted into quadratic function. So P is + definite if its quadratic function is + definite

6. +/- Definite matrix …
A matrix Pnxn (symmetrical or non symmetrical) is - definite
iff 𝑥 𝑇 𝑃𝑥< ∀ 𝑥≠ 0 𝑛×1
In short
If P < 0 => - definite matrix
If 𝑃= 𝑃 𝑇 <0 => Symmetrical - definite matrix
A matrix Pnxn (symmetrical or non symmetrical) is – semi definite
iff 𝑥 𝑇 𝑃𝑥≤ ∀ 𝑥≠ 0 𝑛×1
If P ≤ 0 => - semi definite matrix
If 𝑃= 𝑃 𝑇 ≤0 => Symmetrical - semi definite matrix

7. Test for + definite matrix (Sylvester’s criterion)
Sylvester’s criterion is applicable for symmetrical matrix only
For a symmetric matrix P to be + definite matrix(P>0)
All diagonal elements must be +ve and non zeros(>0).
All the leading principal minors (determinants) must be +ve and non zeros (>0).
Note:
If matrix P is not symmetric matrix, convert it to symmetric matrix by using 𝑃→ 𝑃+ 𝑃 𝑇 2
The leading principal minor of order k of an nxn matrix is obtained by deleting last n – k rows and columns

8. Examples Example 1: Determine the nature of quadratic function
𝑓 𝑥 =7 𝑥 𝑥 1 𝑥 𝑥 1 𝑥 3 +5 𝑥 𝑥 2 𝑥 3 +9 𝑥 3 2
Where 𝑥= 𝑥 1 𝑥 2 𝑥 3 𝑇
Solution:
Above function may be written as
𝑓 𝑥 = 𝑥 1 𝑥 2 𝑥 𝑥 1 𝑥 2 𝑥 3
𝑓 𝑥 = 𝑥 𝑇 𝑃𝑥
Half-Half (To make P symmetrical )
P is symmetrical matrix

9. Example 1… Apply Sylvester’s theorem
Above quadratic function 𝑓 𝑥 = 𝑥 𝑇 𝑃𝑥 is said to be + definite if
𝑓 𝑥 = 𝑥 𝑇 𝑃𝑥> ∀ 𝑥≠ 0 𝑛×1 Or
If P>0 => P is +definite matrix
Apply Sylvester’s theorem
All diagonal elements must be +ve and non zeros. (Satisfied)
All the leading principal minors (determinants) must be +ve and non zeros. (Satisfied)
Leading principal minor of order 1 (by deleting last n-k=3-1=2 row & columns) = 7 > 0
Leading principal minor of order 2 (by deleting last n-k=3-2=1 row & columns) = det =35−4= 31>0
Leading principal minor of order 3 (by deleting last n-k=3-3=0 row & columns) =det > 0

10. Example 1…
Both conditions are satisfied so P is + definite and hence f(x) is + definite quadratic function.

11. Test for + semi definite matrix (Sylvester’s criterion)
For a symmetric matrix P to be + semi definite matrix (P≥0)
All diagonal elements must be +ve and some may be zeros (≥0).
All the principal minors (determinants) must be +ve and some may be zeros (≥0).
(Note: Principal minors not leading principal minors as in + definite matrix )

12. Principal minors of a matrix
Find all the principal minors of matrix P=
Solution
Principal minors of order 1 (=> all 1x1 elements = 9 here but select only diagonal elements) = 7 , 5 & 9
Principal minors of order 2 (=> all 2x2 matrix but select only that matrix whose diagonal elements are also diagonal elements of original matrix)= det , det & det
Principal minors of order 3 (=> matrix itself) = det

13. Example
Example 2: show that following quadratic function is +ve semi definite
𝑓 𝑥 =4 𝑥 1 2 −4 𝑥 1 𝑥 2 + 𝑥 2 2
Solution
Above function can be written as
𝑓 𝑥 = 𝑥 1 𝑥 −2 − 𝑥 1 𝑥 2 = 𝑥 𝑇 𝑃𝑥
For f(x) to be +ve semi definite, 𝑥 𝑇 𝑃𝑥≥ ∀ 𝑥≠ 0 𝑛×1
Or P ≥ 0
(=> P should be +ve semi definite matrix)

14. Example… Apply Sylvester’s theorem
For a symmetric matrix P to be + semi definite matrix
All diagonal elements must be +ve and some may be zeros (= 4 , 1 so satisfied)
All the principal minors (determinants) must be +ve and some may be zeros
Principal minors of order one = 4 , & 1 > 0 (satisfied)
Principal minors of order two = det 4 −2 −2 1 =0 (satisfied)
So P is + semi definite matrix and hence f(x) is + semi definite quadratic function

15. Test for - definite matrix (Sylvester’s criterion)
Sylvester’s criterion is applicable for symmetrical matrix only
For a symmetric matrix Q to be - definite matrix (Q=-P)
All diagonal elements must be -ve and non zeros(<0).
All the leading principal minors (determinants) with even order must be +ve and non zero(>0).
All the leading principal minors (determinants) with odd order must be -ve and non zero(<0).
Note:
Alternate sign: 1st order = -ve, 2nd order = +ve, 3rd order = -ve, …..
If matrix P is + definite then –P will always be –definite matrix.

16. Test for - definite matrix (Sylvester’s criterion)…
Proof:
P > 0 => 𝑥 𝑇 𝑃𝑥> ∀ 𝑥≠ 0 𝑛×1
Multiply both sides with –1
- 𝑥 𝑇 𝑃𝑥<0
𝑥 𝑇 −𝑃 𝑥<0
𝑥 𝑇 𝑄𝑥<0 (Q=-P)
Sign of a determined
det [-P] = (-1)n det [P]
= - det [P] when n is odd
= + det [P] when n is even
Note: All the elements of
+ definite matrix P are multiplied by –ve sign

17. Example
Example 3: show that following quadratic function is –ve definite.
𝑓 𝑥 =−7 𝑥 1 2 −4 𝑥 1 𝑥 2 −10 𝑥 1 𝑥 3 −5 𝑥 2 2 −8 𝑥 2 𝑥 3 −9 𝑥 3 2
Where 𝑥= 𝑥 1 𝑥 2 𝑥 3 𝑇

18. Test for – semi definite matrix (Sylvester’s criterion)
Similar to relation between + definite and – definite matrix
Sylvester’s criterion is applicable for symmetrical matrix only
For a symmetric matrix Q to be – semi definite matrix (Q=-P)
All diagonal elements must be -ve and some may be zeros(≤0).
All the principal minors (determinants) with even order must be +ve and some may be zero (≥0).
All the principal minors (determinants) with odd order must be -ve and some may be zero (≤0).
Note:
Alternate sign: 1st order = -ve, 2nd order = +ve, 3rd order = -ve, …..
If matrix P is + semi definite then –P will always be – semi definite matrix.
Proof can be done as we did in case of – definite matrix from + definite matrix.

19. Example
Example 4: Show that following quadratic function is -ve semi definite
𝑓 𝑥 =−4 𝑥 𝑥 1 𝑥 2 − 𝑥 2 2

20. Next:
Now some other concepts like Optimality conditions.