Vaida Bartkutė, Leonidas Sakalauskas APPLICATION OF ORDER STATISTICS TO TERMINATION OF STOCHASTIC ALGORITHMS Vaida Bartkutė, Leonidas Sakalauskas

Outline Introduction; Application of order statistics to optimality testing and termination of the algorithm: Stochastic Approximation algorithms; Simulated Annealing algorithm; Experimental results; Conclusions.

Outline Introduction; Application of order statistics to optimality testing and termination of the algorithm: Stochastic Approximation algorithms; Simulated Annealing algorithm; Experimental results; Conclusions.

Introduction Termination of the algorithm is a topical problem in stochastic and heuristic optimization. We consider the application of order statistics to establish the optimality in Markov type optimization algorithms. We build a method for the estimation of minima confidence intervals using order statistics, which is implemented for optimality testing and termination.

Statement of the problem The optimization problem is (minimization) as follows: where is a bounded from below locally Lipshitz function. Denote the generalized gradient of this function by Let be the sequence constructed by stochastic search algorithm, where ηt=f(xt), t = 0, 1, …. .

The Markovian algorithms for optimization The Markovian algorithm of random searching represents a Markov chain in which the distribution of probabilities of a point xt+1 depends on a location of the previous point xt and value of function ηt=f(xt) in it, that Examples: Stochastic Approximation; Simulated Annealing; Random Search (Rastrigin method) and etc.

Order statistics and target values for optimality testing and termination Beginning of the problem: Mockus (1968) Theoretical background: Zilinskas, Zhigljavsky (1991) Application to maximum location: Chen (1996) Time-to-target-solution value: Aiex, Resende, & Ribeiro, (2002), Pardalos (2005).

Method for optimality testing by order statistics We build a method for estimation of minimum M of the objective function using values of the function provided in optimization: Let only k+1 order statistics from the sample H to be chosen: , where .

Let apply linear estimators for estimation of the minimum: where . We examine a simple set (Hall (1982)): Let apply linear estimators for estimation of the minimum: where . We examine a simple set (Hall (1982)): Let apply linear estimators for estimation of the minimum: where . We examine a simple set (Hall (1982)):

the objective function is: The one side confidence interval for the minimum value of the objective function is: [ ] where , where  is a confidence level. - the parameter of extreme values distribution; n – dimension; - the parameter of homogeneity of the function f(x) (Zilinskas & Zhigliavsky (1991)).

Stochastic Approximation The smoothing is the standard way for the nondifferentiable optimization. We consider a function smoothed by Lipshitz perturbation operator: where is the value of the perturbation parameter, is a random vector distributed with density p(.). If density p(.) is locally Lipshitz then functions smoothed by this operator are twice continuously differentiable (Rubinstein & Shapiro (1993), Bartkute & Sakalauskas (2004)).

where stochastic gradient, is a scalar multiplier. This scheme is the same for different Stochastic Approximation algorithms whose distinguish only by approach to stochastic gradient estimation. The minimizing sequence converges a.s. to solution of the optimization problem under conditions typical for SA algorithms (Ermolyev (1976), Mikhalevitch et at (1987), Spall (1992), Bartkute & Sakalauskas (2004)).

ESTIMATE OF STOCHASTIC GRADIENT ALGORITHM ESTIMATE OF STOCHASTIC GRADIENT SPSAL, Lipshitz smoothing density (Bartkute & Sakalauskas (2007)) - uniformly distributed in the unit ball. SPSAU uniformly distributed density in the hypercube (Michalevitch et al (1976), (1987)) - uniformly distributed in the hypercube [-1;1]n. FDSA standard finite differences (Ermoliev (1988), Mikhalevitch et al (1987)) - uniformly distributed in the unit ball, - with zero components except ith , equal to 1. - smoothing parameter.

Rate of Convergence Let consider that the function f(x) has a sharp minimum in the point , in which the algorithm converges when Then where A>0, H>0, K>0 are certain constants, is minimum point of the smoothed function (Sakalauskas, Bartkute (2007)).

Experimental results Unimodal testing functions (SPSAL, SPSAU, FDSA) Generated funkcions with sharp minimum- CB3- Rozen Suzuki- Multiextremal testing functions (Simulated Annealing (SA)) Branin- Beale- Rastrigin-

The samples of T=500 test functions were generated, when The samples of T=500 test functions were generated, when and minimized by SPSA with Lipshitz perturbation. The coefficients of the optimizing sequence were chosen according to convergence conditions (Bartkute & Sakalauskas (2006)):

Testing hypothesis about Pareto distribution If order statistics follows from Weibull distribution, then distributed with respect to Pareto distribution (Žilinskas, Zhigljavsky (1991)): Thus, statistical hypothesis tested: . H0:

Testing hypothesis about Pareto distribution The hypothesis tested by criteria 2 ( ) for various stochastic algorithms (critical value 0,46)

One side confidence interval [ ], =0.95 V. Bartkute, L. Sakalauskas. Application of Order Statistics to termination of Stochastic lgorithms. CWU Workshop, December 10-12, 2007

Confidence bounds of the minimum

Confidence bounds of the hitting probability

Termination criterion of the algorithms To stop the algorithm when minima confidence interval becomes less admissible value :

Number of iterations after the termination of the algorithm

Simulated Annealing Algorithm I. Choose temperature updating function neighborhood size function solution generation density function and initial solution x0 (Yang (2000)). II. Construct the optimizing sequence:

F(x,y) = (1.5-x+xy)2 + (2.25-x+xy2)2 + (2.625-x+xy3)2, Experimental results Let consider results of optimality testing with Beale testing function: F(x,y) = (1.5-x+xy)2 + (2.25-x+xy2)2 + (2.625-x+xy3)2, where search domain: -4.5 ≤ x,y ≤ 4.5. It is known that this function has few local minima and global minimum is 0 at the point (3; 0.5).

Confidence bounds of the minimum

Confidence bounds of the hitting probability

Number of iterations after the termination of the algorithm

Conclusions Linear estimator for minimum has been proposed using theory of order statistics, which was studied by experimental way; Developed procedures are simple and depend only on the parameter of extreme values distribution ; Parameter  is easily estimated using a homogeneity of the objective function or by statistical way; Theoretical considerations and computer examples have shown that we can estimate the confidence interval of a function extremum with an admissible accuracy, when the number of iterations increased; Termination rule using the minimum confidence interval was proposed and implemented to Stochastic Approximation and Simulated Annealing.