Name: Mehrab Khazraei(145061) Title: Penalty or Exterior penalty function method professor Name: Sahand Daneshvar

Penalty or Exterior penalty function method is an approach that deal with the non-linear programming problems, having equality and inequality constraints. This method converts inequality constraints to equalities or into problems having simple bound. Exterior penalty function adds a penalty to the objective function to penalize any violation of the constraint. This procedure generates a sequence infeasible points whose it limits is an optimal solution associated with the original problem.

Methods using penalty functions transform a constrained problem into a single unconstrained problem or into a sequence of unconstrained problems. Assume the following problem having the single constraint h(x) = 0: Minimize f ( x ) subject to h(x) = 0.

It could replace by the following unconstrained problem, where μ > 0 is a large number: Minimize f ( x ) + μh 2 (x) subject to x Є R n obviously must h 2 (x) be close to zero. Now consider the following problem having single inequality constraint g(x) 5 0: Minimize f ( x ) subject to g(x) < O

It is clear that the form f ( x ) + pg 2 (x) is not appropriate, since a penalty will be incurred whether g(x) 0. Needless to say, a penalty is desired only if the point x is not feasible, that is, if g(x)> 0. A suitable unconstrained problem is therefore given by: Minimize f ( x ) + μ maximum (0, g(x)} subject to x Є R n

If g(x) 0, then max{0, g(x)} > 0 and the penalty term μg(x) is realized. However, observe that at points x where g(x) = 0, the foregoing objective function might not be differentiable, even though g is differentiable. If differentiability is desirable in such a case, we could, for example, consider instead a penalty function term of the type μ [max{0, g(x)}] 2.

In general, a suitable penalty function must incur a positive penalty for infeasible points and no penalty for feasible points. If the constraints are of the form g i (x) < 0 for i = 1,..., m and h i (x) = 0 for i = 1,..., l, a suitable penalty function α is defined by α(x)= ∑ φ[g i (x)]+∑ψ[h i (x)]

where φ and ψ are continuous functions satisfying the following: φ(y) = 0 if y 0 if y>0 φ(y)=0 if y=0 and φ(y)>0 if y≠0 Typically, φ and ψ are of the forms φ(y) = [max{O, y }] p ψ(Y) = ׀ y ׀ p

Where μ is a positive integer. Thus, the penalty function α is usually of the form α(x)= ∑ [max{0, g i (x)}] p + ∑ ׀ h i (x) ׀ p

Exterior Penalty Function Methods: Presents and proves an important result that justifies using exterior penalty functions as a means for solving constrained problems. Consider the following primal and penalty problems.

Primal Problem Minimize f(x) subject to g(x) < 0 h(x) = 0 x Є X

Where g is a vector function with components g l,..., g m and h is a vector function with components h 1,..., h l. Here, f, g 1,..., g m, h 1,..., h 1 are continuous functions defined on R n, and X is a nonempty set in R n. The set X might typically represent simple constraints that could easily be handled explicitly, such as lower and upper bounds on the variables.

Penalty Problem Let α be a continuous function of the form that satisfying the properties. The basic penalty function approach attempts to find

SUP Ө(μ) subject to μ > 0 Where Ө(p) = inf{f(x) + μα(x): x E X )} The main theorem of this section states that inf{f(x) : x Є X, g(x) < 0, h(x) = 0} = sup Ө(μ) = lim Ө(μ)

From this result it is clear that we can get arbitrarily close to the optimal objective value of the primal problem by computing Ө(p) for a sufficiently large μ. This result is established in the next theorem, however, the following lemma is needed.

Lemma Suppose that f; g l,..., g,, h,,..., h l are continuous functions on R n, and let X be a nonempty set in R n. Let a be a continuous function on R n given by α(x)= ∑ φ[g i (x)]+∑ψ[h i (x)] and suppose that for each μ, there is an x μ Є X such that Ө (μ) = f ( x μ ) + μ a(x μ ). Then, the following statements hold true:

1. Inf{f(x): x Є X, g(x) sup Ө(μ) where Ө(μ),=inf{f(x) + μα (x): x Є X } and where g is the vector function whose components are g l,..., g m, and h is the vector function whose components are h 1,..., h l.

2. f(x μ ) is a non-decreasing function of μ > 0, Ө(p) is a non-decreasing function of μ, and α (x,) is a non-increasing function of μ.

Initialization Step Let E > 0 be a termination scalar. Choose an initial point x l, a penalty parameter μ 1 > 0, and a scalar B > 1. Let k = 1, and go to the Main Step.

Main Step 1. Starting with X k, solve the following problem: Minimize f ( x ) + μ k α (x) subject to x Є X Let x k+1 be an optimal solution and go to Step 2

2. If μ k α (x k+1 ) < E, stop; otherwise, let μ k+1 = B μ k replace k by k + 1, and go to Step 1.

Example Consider the following problem: Minimize (x l - 2) 4 +(X I - 2x 2 ) 2 subject to x x 2 2 = 0 Note that at iteration k, for a given penalty parameter μ k, the problem to be solved for obtaining x μ k is, using the quadratic penalty function: Minimize (X I - 2) 4 + (x l - 2 x 2 ) 2+ μ k (x 1 - x 2 ) 2

Table summarizes the computations

THANK YOU FOR ATTENTION