1. PRELIMINARY MATHEMATICS
LECTURE 5
Optimization

2. Reading Chiang, A. C. and K. Wainwright (2005) Chapters 9 and 11.
Thomas (1999) Chapters
Jacques (2006) Chapter 5.

3. Local maxima and local minima
In this diagram, A and B represent turning points of the curve y = f(x). A is a maximum turning point and B is a minimum turning point. The value of y at point A is known as the local maxima (maximum) and the value of y at point B is called the local minima (minimum).

4. Local maxima and local minima
Note that the slope of the curve y = f(x), measured by
is 0 at both turning points.

5. Second order condition for local maxima and minima
If we draw the derivative on a graph we can confirm that

6. Points of inflexion
The condition however is a necessary condition for local maxima or minima but not a sufficient condition.
In the function below the sign of at point A is changing from positive through 0 to positive again, so this is neither a maximum nor a minimum turning point, but a point of inflection.

7. Second order condition for local maxima and minima
What distinguishes the two points is the slope/gradient of the derivative function

8. Second order condition for local maxima and minima
At point A, which is the point of local maximum, the slope of the curve is negative, while at point B, which is the point of local minimum, the slope is positive.

9. Second order condition for local maxima and minima
The slope of is measured by the derivative of
known as the second derivative, denoted as

10. Second order condition for local maxima and minima
we can see that at the local maximum on the original curve,
and

11. Second order condition for local maxima and minima
we can see that at the local maximum on the original curve,
and
and at the local minimum,

12. Second order condition for a point of inflection
A point of inflexion can be characterised by

13. Second order condition for a point of inflection
A point of inflexion can be characterised by
However it is also possible to have a local maximum or a local minimum with

14. Second order condition for local maxima and minima
Hence, the obtained second order condition
the local maxima,
and
and local minima,
is a second order sufficient condition but not necessary because it is also possible to have a local maximum or a local minimum with

15. n th order condition for local maxima and minima
n th derivative test for relative extrema or inflection point.
If at a stationary point, continue differentiating
until a non-zero higher-order derivative is obtained.

16. n th order condition for local maxima and minima
n th derivative test for relative extrema or inflection point.
If at a stationary point, continue differentiating
until a non-zero higher-order derivative is obtained.
If when evaluated at the stationary point then
if n is an odd number , we have a point of inflection
if n is an even number AND → a local maximum
if n is an even number AND → a local minimum

17. nth derivative test – an example.
Let us consider y = x 6.
The first order necessary condition is obtained as
and x = 0,
Next check the second order sufficient condition:
Evaluating at x = 0 ,

18. nth derivative test – an example.
Because we cannot exclude the possibility that the function is at a local maximum or local minimum, we carry out the nth derivative test to check if this is relative extrema or inflection points.
Third order derivative: , at x = 0,
Forth order derivative: , at x = 0,
Fifth order derivative: , at x = 0,

19. nth derivative test – an example.
Sixth order derivative:
Because the sixth order derivative is non-zero, whereby the non-zero is obtained at an order which is an even number, and because the non-zero derivative is positive, we have a local minimum.

20. nth derivative test – an example.
This is confirmed by drawing a diagram:

21. Recap: Differentiation of bivariate functions
To recap: How to evaluate stationary points…
Step 1. First order condition
Equate the first derivative to zero and find all stationary points.

22. Recap: Differentiation of bivariate functions
To recap: How to evaluate stationary points…
Step 2. Second order sufficient conditions
Evaluate the second order derivative at each stationary points.
If we have a local maximum
If we have a local minimum

23. Recap: Differentiation of bivariate functions
To recap: How to evaluate stationary points…
Step 3. If at any stationary point the second order derivative is zero, carry on differentiating until a non-zero higher order derivative is found.
If n is an odd number, we have a point of inflection
If n is an even number AND we have a local maximum
if n is an even number AND we have a local minimum

24. Second order partial derivatives
Given a function z = f (x, y), the second order (direct) partial derivative signifies that the function has been partially differentiated with respect to one of the independent variables twice while the other independent variable has been held constant
The notation with a double subscript signifies that the primitive function f is differentiated partially with respect to x twice.
The alternative notation resembles that of but with the partial derivatives symbol.

25. Second order partial derivatives
The cross partial derivatives signifies that the function has been partially differentiated with respect to one of the independent variables and then in turn partially differentiated with respect to another independent variable.
The two cross partial derivatives are identical with each other, as long as the two cross partial derivatives are both continuous (Young’s theorem).

26. Unconstrained optimisation
Recall that for a function of a single variable such as
y = f (x)
any point for which
and
was a local minimum. Similarly, any point for which
was a local maximum.

27. Unconstrained optimisation
Now consider a function of two variables
z = f (x, y)
When does z have a local minimum? Extrapolating our arguments of local minima of a single variable function, we may expect z to have a local minimum when it is not possible to obtain a smaller value of z by making slight changes in the values of x and y.
There are three ways in which we can make ‘slight changes’ in x and y:
(a) we can keep x constant and make y vary;
(b) we can keep y constant and make x vary;
(c) we can make both x and y vary at the same time.
If z has a local minimum at point x = a and y = b, then we should be able to change the values of x and y by any of the methods (a), (b), or (c), and yet not make the value of z smaller.

28. Unconstrained optimisation
Let us first consider case (a). In this case, the value of x is held constant. At a point where
and
we cannot obtain a smaller z by varying y alone.
Similarly, if we consider case (b), when
and
we cannot obtain a smaller z by varying x alone.

29. Unconstrained optimisation
Case (c) is slightly complicated. The condition that must be satisfied here is or

30. Unconstrained optimisation
Hence, the second order conditions are as follows:
A point at which the first partial derivatives are zero is:
a local maximum if
a local minimum if
Note that if then we have neither a
maxima nor a minima but what is called a saddle point

31. In the two-variable case, the total differential is
Determinantal test
In the two-variable case, the total differential is
We can also derive the second-order total differential as

32. In the two-variable case, the total differential is
Determinantal test
In the two-variable case, the total differential is
We can also derive the second-order total differential as

33. If d 2 z > 0 (positive definite), local minima
Determinantal test
If d 2 z > 0 (positive definite), local minima
If d 2 z < 0 (negative definite), local maxima

34. The quadratic form can be expressed in matrix form as:
Determinantal test
Since the variables x and y appear only in squares, this can be seen as a quadratic form. As discussed in lecture 1, the condition for d 2 z to be positive/ negative definite may be stated by use of determinants.
The quadratic form can be expressed in matrix form as:

35. Determinantal test
The determinant with the second order partial derivatives as its elements is called a Hessian determinant.

36. The principal minors in this case are:
Determinantal test
The principal minors in this case are:

37. Hence the second order conditions are:
Determinantal test
Hence the second order conditions are:
d 2 z > 0 (positive definite), iff and ,
then local minima.
d 2 z < 0 (negative definite), iff and ,
then local maxima.

38. If we consider a function of three choice variables,
Determinantal test
If we consider a function of three choice variables,
The Hessian determinant is

39. And the principal minors are
Determinantal test
And the principal minors are

40. d 2 z > 0 (positive definite), iff
Determinantal test
d 2 z > 0 (positive definite),
iff
and , then local minima.
d 2 z < 0 (negative definite),
and , then local maxima.

41. Unconstrained optimisation: application
Application: profit maximisation
Let us consider a firm that is a price taker, produces and sells two goods, goods 1 and goods 2 in perfectly competitive markets, and has a profit function:
What would be the level of output for goods 1 and 2 satisfying the first order condition for maximum profit?

42. Unconstrained optimisation: application
Application: profit maximisation
Let us consider a firm that is a price taker, produces and sells two goods, goods 1 and goods 2 in perfectly competitive markets, and has a profit function:
What would be the level of output for goods 1 and 2 satisfying the first order condition for maximum profit?

43. Unconstrained optimisation: application
Application: profit maximisation
Setting both condition equal to zero,

44. Unconstrained optimisation: application
Application: profit maximisation
Setting both condition equal to zero,
Rewrite as

45. Unconstrained optimisation: application
Application: profit maximisation
Setting both condition equal to zero,
Rewrite as
Express this in terms of matrices

46. Unconstrained optimisation: application
Application: profit maximisation
Use Cramer’s rule

47. Unconstrained optimisation: application
Application: profit maximisation
Let us consider a firm that is a price taker, produces and sells two goods, goods 1 and goods 2 in perfectly competitive markets, and has a profit function:
What would be the level of output for goods 1 and 2 satisfying the first order condition for maximum profit?
The level of output/supply for good 1 must be 40 and for good 2 be 24 in order to satisfy the first order conditions.

48. Unconstrained optimisation: application
Application: profit maximisation
Let us consider a firm that is a price taker, produces and sells two goods, goods 1 and goods 2 in perfectly competitive markets, and has a profit function:
What would be the level of output for goods 1 and 2 satisfying the first order condition for maximum profit?
The level of output/supply for good 1 must be 40 and for good 2 be 24 in order to satisfy the first order conditions.
Next, check the second order conditions to verify if we have local maxima.

49. Unconstrained optimisation: application
Application: profit maximisation
Let us consider a firm that is a price taker, produces and sells two goods, goods 1 and goods 2 in perfectly competitive markets, and has a profit function:
Let us consider whether the stationary point we found at
Q1 = 40 and Q2 = 24 is a local maxima or a local minima.

50. Unconstrained optimisation: application
Application: profit maximisation
The Hessian matrix can be written as
Hence the profit is at local maximum at Q1 = 40 and Q2 = 24