Introduction to Optimization 01-Oct-13 Dr. Walid Al-Awad

Introduction to Optimization A structure in mechanics is defined as “any assemblage of materials which is intended to sustain loads.” Optimization means making things the best. Optimization can be defined as the process of finding the conditions that give the maximum or minimum of a function. 01-Oct-13 Dr. Walid Al-Awad

Introduction to Optimization Many times in life we are asked to do an optimization problem - that is, find the largest or smallest value of some quantity that will fulfill a need. Typical situations are: find the route which will minimize the time it takes me to get to university. build a structure using the least amount of material. build a structure costing the least amount of money. build a yard enclosing the most amount of space. Design a beam using the smallest dimension 01-Oct-13 Dr. Walid Al-Awad

Introduction to Optimization they are all trying to maximize or minimize some quantity In every optimization problem, you are always looking for a quantity to be maximized or minimized, like - minimize area , - smallest volume , - least amount of time , - shortest distance , - cheapest price. 01-Oct-13 Dr. Walid Al-Awad

Introduction to Optimization A discipline, optimization is often called mathematical programming. (mathematical optimization (alternatively, optimization or mathematical programming) is the selection of a best element (with regard to some criteria) from some set of available alternatives) 01-Oct-13 Dr. Walid Al-Awad

Determining the root Recall, when determining the root, we were seeking x where f(x) = 0 With optimization, however we are seeking f '(x) = 0 01-Oct-13 Dr. Walid Al-Awad

Maximum, Minimum The Maximum occurs when f "(x)<0 The Minimum occurs when f "(x)>0 01-Oct-13 Dr. Walid Al-Awad

Optimization In some techniques, we determine the optima by solving the root problem: f '(x) =0 If f '(x) is not available analytically, we may use a finite difference approximation to estimate the derivative 01-Oct-13 Dr. Walid Al-Awad

Example max, min Determine the maximum and minimum values of the function: Solution: Since f `(x)=60(x4-3x3+2x2)=60x2(x-1)(x-2), f `(x)=0 at x=0,x=1, and x=2. The second derivative is: At x=1, f’’(x)=-60 and hence x=1 is a relative maximum. Therefore, fmax= f (x=1) = 12 At x=2, f’’(x)=240 and hence x=2 is a relative minimum. Therefore, fmin= f (x=2) = -11 01-Oct-13 Dr. Walid Al-Awad

Example max, min Solution cont’d: At x=0, f’’(x)=0 and hence we must investigate the next derivative. Since at x=0, x=0 is neither a maximum nor a minimum, and it is an inflection point. 01-Oct-13 Dr. Walid Al-Awad

f’(x) exists everywhere in [0,5] f(x)= x2 -2x -5 f’(x) = 2x – 2 f’(x) = 0 at x=1 f’(x) exists everywhere in [0,5] So the critical point are x=0, x=1, x=5 f(0) = (0)2 – 2(0) -5 = -5 f(1) (1)2 – 2(1) -5 =-6 f(5) = (5)2 – 2(5) -5 = 10 The minimum value of f(x) occurs at x=1 and the maximum value at x=5. 01-Oct-13 Dr. Walid Al-Awad

max, min 01-Oct-13 Dr. Walid Al-Awad

max, min - Search for Two Variables Find the values of x and z (both > 0) that maximize Solution: Formulation as an optimization problem: Design variables: x, z positive Solving the two equations for x and z gives: x = 9.333 and z = 8.667 With these values, U = 83.33 01-Oct-13 Dr. Walid Al-Awad

GUIDELINES FOR SOLVING MAX./MIN. PROBLEMS GUIDELINES OR (STEPS )FOR SOLVING MAX./MIN. PROBLEMS Read each problem slowly and carefully. Read the problem at least three times before trying to solve it. If appropriate, draw a sketch or diagram of the problem to be solved. Pictures are a great help in organizing and sorting out your thoughts. Define variables to be used and carefully label your picture or diagram with these variables. This step is very important because it leads directly or indirectly to the creation of mathematical equations. 01-Oct-13 Dr. Walid Al-Awad

MAX./MIN. PROBLEMS 4. Write down all equations which are related to your problem or diagram. Clearly denote that equation which you are asked to maximize or minimize. 5. Before differentiating, make sure that the optimization equation is a function of only one variable. Then differentiate using the well-known rules of differentiation. 6. Verify that your result is a maximum or minimum value using the first or second derivative test for extrema. 01-Oct-13 Dr. Walid Al-Awad

MAX./MIN. PROBLEMS PROBLEM 1 : Find two nonnegative numbers whose sum is 9 and so that the product of one number and the square of the other number is a maximum. SOLUTION 1: Let variables x and y represent two nonnegative numbers. The sum of the two numbers is given to be 9 = x + y , so that y = 9 - x . We wish to MAXIMIZE the PRODUCT P = x y2 . However, before we differentiate the right-hand side, we will write it as a function of x only. Substitute for y getting P = x y2 = x ( 9-x)2 . 01-Oct-13 Dr. Walid Al-Awad

is the largest possible product. MAX./MIN. PROBLEMS Now differentiate this equation using the product rule and chain rule, getting P = x ( 9-x)2 = x(81 -18x +x2) = x3 -18*x2 +81x P ́ = 3*x -18*2*x +81 =0 P ́ = 0 for x=9 or x=3 . Note that since both x and y are nonnegative numbers and their sum is 9, it follows that 0≤ x≤ 9 . See the adjoining sign chart for P' . If x=3 and y=6 , P=3*62 P=108 is the largest possible product. 01-Oct-13 Dr. Walid Al-Awad

MAX./MIN. PROBLEMS PROBLEM 2 : Build a rectangular pen with parallel partitions using 500 feet of fencing. What dimensions will maximize the total area of the pen ? SOLUTION 2 : Let variable x be the width of the pen and variable y the length of the pen The total amount of fencing is given to be 500 = 5 (width) + 2 (length) = 5x + 2y , 2y = 500 - 5x y = 250 - (5/2)x 01-Oct-13 Dr. Walid Al-Awad

MAX./MIN. PROBLEMS We wish to MAXIMIZE the total AREA of the pen A = (width) (length) = x y . before we differentiate the right-hand side, we will write it as a function of x only. Substitute for y getting A = x y = x ( 250 - (5/2)x) = 250x - (5/2)x2 . Now differentiate this equation, getting A' = 250 - (5/2) 2x = 250 - 5x = 5 (50 - x ) = 0 x=50 . 01-Oct-13 Dr. Walid Al-Awad

See the adjoining sign chart for A' . x=50 ft. and y=125 ft. , MAX./MIN. PROBLEMS Note that since there are 5 lengths of x in this construction and 500 feet of fencing, it follows that 0≤ x ≤100. See the adjoining sign chart for A' . x=50 ft. and y=125 ft. , A = 6250 ft.2 is the largest possible area of the pen. 01-Oct-13 Dr. Walid Al-Awad

MAX./MIN. PROBLEMS PROBLEM 3 : An open rectangular box with square base is to be made from 48 ft.2 of material. What dimensions will result in a box with the largest possible volume ? SOLUTION 3 : Let variable x be the length of one edge of the square base and variable y the height of the box. The total surface area of the box is given to be 48 = (area of base) + 4 (area of one side) = x2 + 4 (xy) , 01-Oct-13 Dr. Walid Al-Awad

MAX./MIN. PROBLEMS 4xy = 48 - x2 We wish to MAXIMIZE the total VOLUME of the box V = (length) (width) (height) = (x) (x) (y) = x2 y . before we differentiate the right-hand side, we will write it as a function of x only. Substitute for y getting V = x2 y = 12x - (1/4)x3 . Now differentiate this equation, getting V' = 12 - (1/4)3x2 = 12 - (3/4)x2 = (3/4)(16 - x2 ) = (3/4)(4 - x)(4 + x) = 0 01-Oct-13 Dr. Walid Al-Awad

is the largest possible volume of the box. MAX./MIN. PROBLEMS x=4 or x=-4 . But x ≠ -4 since variable x measures a distance and x > 0 . Since the base of the box is square and there are 48 ft.2 of material, it follows that . See the adjoining sign chart for V' . x=4 ft. and y=2 ft. , V = 32 ft.3 is the largest possible volume of the box. 01-Oct-13 Dr. Walid Al-Awad

Example max, min Problem 4: A rectangle has a perimeter of 71 feet. What length and width should it have so that its area is a maximum? What is this maximum area? Area = x * y 2x + 2y =71 y= 35.5 – x f(x)= Area = x * (35.5 –x) = 35.5 x –x2 f `(x)=0 35.5 -2x =0 x= 17.25 ft y=17.25ft Maximum area = 315.063 ft2 01-Oct-13 Dr. Walid Al-Awad

max, min Example 01-Oct-13 Dr. Walid Al-Awad

max, min 01-Oct-13 Dr. Walid Al-Awad

max, min 01-Oct-13 Dr. Walid Al-Awad

max, min 01-Oct-13 Dr. Walid Al-Awad

ENGINEERING APPLICATIONS OF OPTIMIZATION Optimization can be applied to solve any engineering problem. Some typical applications: Design of aircraft and aerospace structures for minimum weight. Design of civil engineering structures such as frames, foundations, bridges, towers, and dams for minimum cost. Minimum-weight design of structures for earthquake, wind, and other types of random loading. Design of water resources systems for maximum benefit 01-Oct-13 Dr. Walid Al-Awad

ENGINEERING APPLICATIONS OF OPTIMIZATION Design of pumps, turbines, and heat transfer equipment for maximum efficiency Optimum design of electrical machinery such as motors, generators, and transformers Optimal production planning, controlling, and scheduling Design of optimum pipeline networks for process industries Optimum design of control systems Optimum design of chemical processing Analysis of statistical data and building empirical models from experimental results to obtain the most accurate representation of the physical phenomenon 01-Oct-13 Dr. Walid Al-Awad

Optimization STATEMENT OF AN OPTIMIZATION PROBLEM An optimization problem can be stated as follows. A real function of n variables (1) Without these, we have unconstrained optimization With them, we have constrained optimization 01-Oct-13 Dr. Walid Al-Awad

Optimization where X is an n-dimensional vector called the design vector, f (X) is termed the objective function, and gj (X) and lj (X) are known as inequality and equality constraints, respectively. The number of variables n and the number of constraints m and/or p need not be related in any way. The problem stated in Eq. ( 1) is called a constrained optimization problem. Some optimization problems do not involve any constraints and can be stated as Eq. ( 2) 01-Oct-13 Dr. Walid Al-Awad

Optimization Such problems are called unconstrained optimization problems. (2) Design Vector :Any engineering system or component is defined as variables during the design process. Variables: called design or decision variables xi , i = 1, 2, . . . , n. 01-Oct-13 Dr. Walid Al-Awad

Optimization constraints, Unconstrained Optimization with constraints Unconstrained optimization 01-Oct-13 Dr. Walid Al-Awad

Optimization Procedure 01-Oct-13 Dr. Walid Al-Awad

Optimization Basic Ideas One-dimensional Unconstrained Optimization - Golden-Section Search - Newton’s Method Multidimensional Unconstrained Optimization - Direct Methods - Gradient Methods 01-Oct-13 Dr. Walid Al-Awad

Golden-Section Search A unimodal function has a single maximum or a minimum in the a given interval. For a unimodal function: First pick two points that will bracket your extremism [xl, xu]. Pick an additional third point within this interval to determine whether a maximum occurred. Then pick a fourth point to determine whether the maximum has occurred within the first three or last three points The key is making this approach efficient by choosing intermediate points wisely thus minimizing the function evaluations by replacing the old values with new values. 01-Oct-13 Dr. Walid Al-Awad

Golden-Section Search The first condition specifies that the sum of the two sub lengths l1 and l2 must equal the original interval length. The second say that the ratio of the length must be equal 01-Oct-13 Dr. Walid Al-Awad

Golden-Section Search consider the Golden Section Search method which is based on the Golden Ratio The Parthenon (Ancient Greek: Παρθενών) is a temple in the Athenian Acropolis, Greece, dedicated to of the Greek goddess Athena, whom the people of Athens considered their protector. Its construction began in 447 BC and was completed in 438 BC. The Parthenon 5th century BC The Golden Rectangle A rectangle is called a golden rectangle if the ratio of the sides of the rectangle is equal to 0.61803, like the one shown . 0.61803 1 01-Oct-13 Dr. Walid Al-Awad

Golden-Section Search Golden Ratio and Fibonacci Numbers 0,1,1,2,3,5,8,13,21,34….. 0/1=0 1/1=1 1/2=0.5 2/3=0.667 3/5==0.6 5/8=0.625 8/13=0.615 Continue and the ratio approaches the golden ratio! 01-Oct-13 Dr. Walid Al-Awad

Golden-Section Search Start by defining an interval contains single answer, which is called unimodal interval. Find the two interior points x1,x2 d=0.618 *( xU –xL) x1 = xL + d x2 = xU – d If f(x1)≥ f(x2) xopt =x1, xL =x2 If (x2) > f(x1) xopt=x2, xU =x1 01-Oct-13 Dr. Walid Al-Awad

Golden-section Search Initial Step of the Golden-section Search 1) Guess initial bracket xL and xU 2) Choose two interior points x1 and x2 according to golden ratio, 3) If f (x1) > f (x2), eliminate [ xL , x2 ] and set x2 = xL for next round 01-Oct-13 Dr. Walid Al-Awad

Golden-section Search 4) Only new x1 need to be determined, 01-Oct-13 Dr. Walid Al-Awad

Example: Golden-Section Search to find maximum Solution: (1) Create two interior points (2) Evaluate function at interior points, 01-Oct-13 Dr. Walid Al-Awad

Example: Golden-Section Search to find maximum (3) Because f(x2) > f(x1), eliminate upper part, (4) Compute new x2 (5) Evaluate function at x2 01-Oct-13 Dr. Walid Al-Awad

Example: Golden-Section Search to find maximum 01-Oct-13 Dr. Walid Al-Awad

Golden-Section Search Example Consider Figure below. The cross-sectional area A of a gutter with equal base and edge length of 2 is given by A= 4 sinθ(1+cosθ) Find the angle θ which maximizes the cross-sectional area of the gutter. Using an initial interval of [0,π/2], find the solution after 2 iterations. Use an initial ε=0.05. Cross section of the gutter 01-Oct-13 Dr. Walid Al-Awad

Golden-Section Search Solution: The function to be maximized is f(θ)=4sinθ (1+cosθ) Iteration 1: Given the values for the boundaries of xl=0 and xu=π /2, we can calculate the initial intermediate points as follows: x1=xl + (√5 -1)/2 *(xu – xl ) = (0 + √5 -1)/2 * 1.5708 = 0.9708 x2 = xu+ (√5 -1)/2 *(xu – xl )= (1.578 - √5 -1)/2*1.5708 = 0.60 f(0.6) 4.1227, f(0.9708)=5.165 f(x1) > f(x2) xl=x2 x1=xl + (√5 -1)/2 *(xu – xl )=0.6 + (√5 -1)/2(1.57-0.6) = 1.2 01-Oct-13 Dr. Walid Al-Awad

Golden-Section Search To check the stopping criteria the difference between xu and xl is calculated to be xu –xl = 1.57 -0.6 =0.97 which is greater than ε = 0.05. The process is repeated in the second iteration. Iteration 2: The values for the boundary and intermediate points used in this iteration were calculated in the previous iteration as shown below. xl= 0.6 xu=1.57 x1=1.2 x2=0.97 01-Oct-13 Dr. Walid Al-Awad

Golden-Section Search 01-Oct-13 Dr. Walid Al-Awad

Golden-Section Search Example By using the golden section search method, find the value of x that maximize f (x) = -1.5x6 - 2x4 + 12x Over the interval xl=0,xu=2, perform three iterations only 01-Oct-13 Dr. Walid Al-Awad

Golden-Section Search Use the golden section search method to find the value of x that maximize x0=-2, xu=4, εs=1% f (x) = 4x -1.8x2 + 1.2x3 - 0.3x4 01-Oct-13 Dr. Walid Al-Awad

Newton- Raphson method A similar approach to Newton- Raphson method can be used to find an optimum of f(x) by defining a new function g(x)=f’(x). Thus because the same optimal value x* satisfies both f‘(x*)=g(x*)=0. We can use the following as a technique to the extremism of f(x). 01-Oct-13 Dr. Walid Al-Awad

Newton’s Method Newton’s Method 01-Oct-13 Dr. Walid Al-Awad

Newton’s Method 01-Oct-13 Dr. Walid Al-Awad

Newton’s Method 01-Oct-13 Dr. Walid Al-Awad

Newton’s Method Use Newton”s method to minimize f(x). x0 = 1 f (x) = 0.5 – x e-x2 f(x)’ = (2x2 -1) * e-x2 f’’(x) = 2 x (3 – 2 x2) * e-x2 xi+1 = xi – f’(x) / f’’(x) = xi – ( 2xi2 -1)/(2xi(3-2xi2 )) I xi f(xi) 1 0.132 0.5 0.111 2 0.7 0.071 3 0.707 01-Oct-13 Dr. Walid Al-Awad

Optimization The End 01-Oct-13 Dr. Walid Al-Awad