1. August, 2005 Department of Computer Science, University of Ioannina Ioannina - GREECE 1 To STOP or not to STOP By I. E. Lagaris A question in Global Optimization

2. August, 2005 Department of Computer Science, University of Ioannina Ioannina - GREECE 2 Contributions Research performed in collaboration with Ioannis G. Tsoulos .  PhD candidate, Dept. of CS, Univ. of Ioannina

3. August, 2005 Department of Computer Science, University of Ioannina Ioannina - GREECE 3 Searching for “Local Minima” One-Dimensional Example Exhaustive procedure: From left to right minimization-maximization repetition.

4. August, 2005 Department of Computer Science, University of Ioannina Ioannina - GREECE 4 Searching for “Local Minima” Two-Dimensional Example “Egg holder” The exhaustive technique used in one-dimension, is not applicable in two or more dimensions. Level plots in 2-D

5. August, 2005 Department of Computer Science, University of Ioannina Ioannina - GREECE 5 The “ MULTISTART ” algorithm  Sample a point x from S  Start a local search, leading to a minimum y  If y is a new minimum, add it to the list of minima  Decide “ to STOP or not to STOP ”  Repeat If the decision is right, the iterations will not stop before all minima inside the bounded domain S are found.

6. August, 2005 Department of Computer Science, University of Ioannina Ioannina - GREECE 6 The “Region of Attraction” (RA)  The set of all points that when a local search is started from, concludes to the same minimum.  Formally:  The RA depends strongly on the local search (LS) procedure.  The measure of an RA is denoted by m(A i ).

7. August, 2005 Department of Computer Science, University of Ioannina Ioannina - GREECE 7 ASSUMPTIONS …  Deterministic local search. Implies non-overlapping basins.  Sampling is based on the uniform distribution. Implies that a sampled point belongs to A i with probability:  There is no zero-measure basin, i.e.

8. August, 2005 Department of Computer Science, University of Ioannina Ioannina - GREECE 8 Coverage based stopping rule i.e.: STOP when c →1 w, being the number of minima discovered so far. If can be calculated, then a rule may be formulated based on the space coverage:

9. August, 2005 Department of Computer Science, University of Ioannina Ioannina - GREECE 9 Estimating m(A i ) Let L be the number of the performed local searches and L i those that ended at y i. An estimation then, may be obtained by: Unfortunately this estimation is useless in the present framework, since it will always yield: c = 1 note that:

10. August, 2005 Department of Computer Science, University of Ioannina Ioannina - GREECE 10 Double Box Consider a box S 2 that contains S and satisfies: Sample points from S 2, and perform local searches only from points contained in S. L, now stands for the total number of sampled points.

11. August, 2005 Department of Computer Science, University of Ioannina Ioannina - GREECE 11 Implementation  Keep sampling from S 2 until N points in S are collected. ( N =1 for Multistart)  At iteration k, let M k be the total number of sampled points ( kN of them in S ). and → 1 → 0 last indicates the iteration during which the latest minimum was discovered  STOP if: and

12. August, 2005 Department of Computer Science, University of Ioannina Ioannina - GREECE 12 FunctionMinimaCallsMinimaCallsMinimaCallsMinimaCalls Shubert4001150243400577738400322447395139768 Gkls(3,30)30961269293025832341026153920 Rastrigin4950384491959349135814910034 Test2N(5)3278090323060732208703213462 Test2N(6)6485380643484064225356415393 Guilin(20)100340511210019062881008545117179331 Shekel-101093666103683810237801015976 p0.30.50.70.9 Multistart performance with Double Box, for a range of p values

13. August, 2005 Department of Computer Science, University of Ioannina Ioannina - GREECE 13 Observables rule  This rule relies on the agreement of values of observable (i.e. measurable) quantities, to their expected asymptotic values.  The number of times L i that minimum y i is found, is compared to its expected value.  y i are indexed in order of their appearance. Hence y 1 requires one application of the LS, y 2 requires additional n 2 applications, y 3 additional n 3 …  Let the number of the recovered minima so far be denoted by w.

14. August, 2005 Department of Computer Science, University of Ioannina Ioannina - GREECE 14 Expectation values The expectation value of the number of times the i th minimum is found, at the time when the w th minimum is recovered for the first time, is recursively given by: An estimation that may be used is:

15. August, 2005 Department of Computer Science, University of Ioannina Ioannina - GREECE 15 Keep trying … Suppose that after having found w minima, there is a number (say K) of consecutive trials without any success, i.e. without finding a new minimum. The expected number of times the i th minimum is found at that moment is given recursively by:

16. August, 2005 Department of Computer Science, University of Ioannina Ioannina - GREECE 16 The Observables’ criterion The quantity: Tends asymptotically to zero. Hence, STOP if:

17. August, 2005 Department of Computer Science, University of Ioannina Ioannina - GREECE 17 “Expected Minimizers” Rule  Based on estimating the number of local minima inside the domain of interest.  The estimation is improving as the algorithm proceeds.  The key quantity is the probability that l minima are found after m trials.  Calculated recursively.

18. August, 2005 Department of Computer Science, University of Ioannina Ioannina - GREECE 18 Probabilities If stands for the probability to recover in a single trial, then the probability of finding l minima in m trials is given by: Probability that a new minimum is found other than Probability that one of the first l minima is found again. Note that:

19. August, 2005 Department of Computer Science, University of Ioannina Ioannina - GREECE 19 Expected values The expected value for the number of minima, estimated after m trials is given by: The corresponding variance is given by: We use the estimate: STOP if : The RULE

20. August, 2005 Department of Computer Science, University of Ioannina Ioannina - GREECE 20 Other rules Uncovered fraction of space: Zieliński (1981) STOP if: Estimated number of minima: Boender & Rinnooy Kan (1987) STOP if: Probability all minima are found: Boender & Romeijn (1995) STOP if:

21. August, 2005 Department of Computer Science, University of Ioannina Ioannina - GREECE 21 Uncovered fraction Estimated # of minima Double BoxObservablesExpected # of minima MULTISTART

22. August, 2005 Department of Computer Science, University of Ioannina Ioannina - GREECE 22 Uncovered fraction Estimated # of minima Double BoxObservablesExpected # of minima TMLSL

23. August, 2005 Department of Computer Science, University of Ioannina Ioannina - GREECE 23 Conclusions  The new rules improve the performance at least for problems in our benchmark suite.  Proper choice of the parameter p, for different methods is important.  Remains to be seen if performance is also boosted in other practical applications.