1. Vaida Bartkutė, Leonidas Sakalauskas
APPLICATION OF ORDER STATISTICS TO TERMINATION OF STOCHASTIC ALGORITHMS
Vaida Bartkutė,
Leonidas Sakalauskas

2. Outline Introduction;
Application of order statistics to optimality testing and termination of the algorithm:
Stochastic Approximation algorithms;
Simulated Annealing algorithm;
Experimental results;
Conclusions.

3. Outline Introduction;
Application of order statistics to optimality testing and termination of the algorithm:
Stochastic Approximation algorithms;
Simulated Annealing algorithm;
Experimental results;
Conclusions.

4. Introduction
Termination of the algorithm is a topical problem in stochastic and heuristic optimization.
We consider the application of order statistics to establish the optimality in Markov type optimization algorithms.
We build a method for the estimation of minima confidence intervals using order statistics, which is implemented for optimality testing and termination.

5. Statement of the problem
The optimization problem is (minimization) as follows:
where is a bounded from below locally Lipshitz
function. Denote the generalized gradient of this function by
Let be the sequence constructed by
stochastic search algorithm, where ηt=f(xt), t = 0, 1, …. .

6. The Markovian algorithms for optimization
The Markovian algorithm of random searching represents a Markov chain in which the distribution of probabilities of a point xt+1 depends on a location of the previous point xt and value of function ηt=f(xt) in it, that
Examples: Stochastic Approximation;
Simulated Annealing;
Random Search (Rastrigin method) and etc.

7. Order statistics and target values for optimality testing and termination
Beginning of the problem:
Mockus (1968)
Theoretical background:
Zilinskas, Zhigljavsky (1991)
Application to maximum location:
Chen (1996)
Time-to-target-solution value:
Aiex, Resende, & Ribeiro, (2002),
Pardalos (2005).

8. Method for optimality testing by order statistics
We build a method for estimation of minimum M of
the objective function using values of the function provided in
optimization:
Let only k+1 order statistics from the sample H to be chosen: ,
where

9. Let apply linear estimators for estimation of the minimum:
where
We examine a simple set (Hall (1982)):
Let apply linear estimators for estimation of the minimum:
where
We examine a simple set (Hall (1982)):
Let apply linear estimators for estimation of the minimum:
where
We examine a simple set (Hall (1982)):

10. the objective function is:
The one side confidence interval for the minimum value of
the objective function is:
[ ]
where , where  is a confidence level.
- the parameter of extreme values distribution;
n – dimension;
- the parameter of homogeneity of the function f(x) (Zilinskas & Zhigliavsky (1991)).

11. Stochastic Approximation
The smoothing is the standard way for the nondifferentiable
optimization. We consider a function smoothed by Lipshitz
perturbation operator:
where is the value of the perturbation parameter,
is a random vector distributed with density p(.).
If density p(.) is locally Lipshitz then functions smoothed by this operator are twice continuously differentiable (Rubinstein & Shapiro (1993), Bartkute & Sakalauskas (2004)).

12. where stochastic gradient,
is a scalar multiplier.
This scheme is the same for different Stochastic Approximation algorithms whose distinguish only by approach to stochastic gradient estimation. The minimizing sequence converges a.s. to solution of the optimization problem under conditions typical for SA algorithms (Ermolyev (1976), Mikhalevitch et at (1987), Spall (1992), Bartkute & Sakalauskas (2004)).

13. ESTIMATE OF STOCHASTIC GRADIENT
ALGORITHM
ESTIMATE OF STOCHASTIC GRADIENT
SPSAL,
Lipshitz smoothing density (Bartkute & Sakalauskas (2007))
- uniformly distributed in the unit ball.
SPSAU
uniformly distributed density in the hypercube
(Michalevitch
et al (1976), (1987))
- uniformly distributed in the hypercube [-1;1]n.
FDSA
standard finite differences
(Ermoliev (1988), Mikhalevitch et al (1987))
- uniformly distributed in the unit ball,
- with zero components except ith , equal to 1.
- smoothing parameter.

14. Rate of Convergence Let consider that the function f(x) has a sharp
minimum in the point , in which the algorithm converges
when
Then
where A>0, H>0, K>0 are certain constants, is minimum point of the smoothed function (Sakalauskas, Bartkute (2007)).

15. Experimental results Unimodal testing functions
(SPSAL, SPSAU, FDSA)
Generated funkcions with sharp minimum-
CB3-
Rozen Suzuki-
Multiextremal testing functions
(Simulated Annealing (SA))
Branin-
Beale-
Rastrigin-

16. The samples of T=500 test functions were generated, when
The samples of T=500 test functions were generated, when and minimized by SPSA with Lipshitz perturbation.
The coefficients of the optimizing sequence were chosen according to convergence conditions (Bartkute & Sakalauskas (2006)):

17. Testing hypothesis about Pareto distribution
If order statistics follows from Weibull distribution, then
distributed with respect to Pareto distribution (Žilinskas, Zhigljavsky (1991)):
Thus, statistical hypothesis tested: .
H0:

18. Testing hypothesis about Pareto distribution
The hypothesis tested by criteria 2 ( ) for various stochastic algorithms (critical value 0,46)

19. One side confidence interval
[ ],
=0.95
V. Bartkute, L. Sakalauskas. Application of Order Statistics to termination of Stochastic lgorithms CWU Workshop, December 10-12, 2007

20. Confidence bounds of the minimum

21. Confidence bounds of the hitting probability

22. Termination criterion of the algorithms
To stop the algorithm when minima confidence interval
becomes less admissible value :

23. Number of iterations after the termination
of the algorithm

24. Simulated Annealing Algorithm
I. Choose temperature updating function neighborhood size
function solution generation density function
and initial solution x0 (Yang (2000)).
II. Construct the optimizing sequence:

25. F(x,y) = (1.5-x+xy)2 + (2.25-x+xy2)2 + (2.625-x+xy3)2,
Experimental results
Let consider results of optimality testing with Beale testing function:
F(x,y) = (1.5-x+xy)2 + (2.25-x+xy2)2 + (2.625-x+xy3)2,
where search domain: -4.5 ≤ x,y ≤ 4.5.
It is known that this function has few local minima and
global minimum is 0 at the point (3; 0.5).

26. Confidence bounds of the minimum

27. Confidence bounds of the hitting probability

28. Number of iterations after the termination
of the algorithm

29. Conclusions
Linear estimator for minimum has been proposed using theory of order statistics, which was studied by experimental way;
Developed procedures are simple and depend only on the parameter of extreme values distribution ;
Parameter  is easily estimated using a homogeneity of the objective function or by statistical way;
Theoretical considerations and computer examples have shown that we can estimate the confidence interval of a function extremum with an admissible accuracy, when the number of iterations increased;
Termination rule using the minimum confidence interval was proposed and implemented to Stochastic Approximation and Simulated Annealing.