August, 2005 Department of Computer Science, University of Ioannina Ioannina - GREECE 1 To STOP or not to STOP By I. E. Lagaris A question in Global Optimization

August, 2005 Department of Computer Science, University of Ioannina Ioannina - GREECE 2 Contributions Research performed in collaboration with Ioannis G. Tsoulos .  PhD candidate, Dept. of CS, Univ. of Ioannina

August, 2005 Department of Computer Science, University of Ioannina Ioannina - GREECE 3 Searching for “Local Minima” One-Dimensional Example Exhaustive procedure: From left to right minimization-maximization repetition.

August, 2005 Department of Computer Science, University of Ioannina Ioannina - GREECE 4 Searching for “Local Minima” Two-Dimensional Example “Egg holder” The exhaustive technique used in one-dimension, is not applicable in two or more dimensions. Level plots in 2-D

August, 2005 Department of Computer Science, University of Ioannina Ioannina - GREECE 5 The “ MULTISTART ” algorithm  Sample a point x from S  Start a local search, leading to a minimum y  If y is a new minimum, add it to the list of minima  Decide “ to STOP or not to STOP ”  Repeat If the decision is right, the iterations will not stop before all minima inside the bounded domain S are found.

August, 2005 Department of Computer Science, University of Ioannina Ioannina - GREECE 6 The “Region of Attraction” (RA)  The set of all points that when a local search is started from, concludes to the same minimum.  Formally:  The RA depends strongly on the local search (LS) procedure.  The measure of an RA is denoted by m(A i ).

August, 2005 Department of Computer Science, University of Ioannina Ioannina - GREECE 7 ASSUMPTIONS …  Deterministic local search. Implies non-overlapping basins.  Sampling is based on the uniform distribution. Implies that a sampled point belongs to A i with probability:  There is no zero-measure basin, i.e.

August, 2005 Department of Computer Science, University of Ioannina Ioannina - GREECE 8 Coverage based stopping rule i.e.: STOP when c →1 w, being the number of minima discovered so far. If can be calculated, then a rule may be formulated based on the space coverage:

August, 2005 Department of Computer Science, University of Ioannina Ioannina - GREECE 9 Estimating m(A i ) Let L be the number of the performed local searches and L i those that ended at y i. An estimation then, may be obtained by: Unfortunately this estimation is useless in the present framework, since it will always yield: c = 1 note that:

August, 2005 Department of Computer Science, University of Ioannina Ioannina - GREECE 10 Double Box Consider a box S 2 that contains S and satisfies: Sample points from S 2, and perform local searches only from points contained in S. L, now stands for the total number of sampled points.

August, 2005 Department of Computer Science, University of Ioannina Ioannina - GREECE 11 Implementation  Keep sampling from S 2 until N points in S are collected. ( N =1 for Multistart)  At iteration k, let M k be the total number of sampled points ( kN of them in S ). and → 1 → 0 last indicates the iteration during which the latest minimum was discovered  STOP if: and

August, 2005 Department of Computer Science, University of Ioannina Ioannina - GREECE 12 FunctionMinimaCallsMinimaCallsMinimaCallsMinimaCalls Shubert Gkls(3,30) Rastrigin Test2N(5) Test2N(6) Guilin(20) Shekel p Multistart performance with Double Box, for a range of p values

August, 2005 Department of Computer Science, University of Ioannina Ioannina - GREECE 13 Observables rule  This rule relies on the agreement of values of observable (i.e. measurable) quantities, to their expected asymptotic values.  The number of times L i that minimum y i is found, is compared to its expected value.  y i are indexed in order of their appearance. Hence y 1 requires one application of the LS, y 2 requires additional n 2 applications, y 3 additional n 3 …  Let the number of the recovered minima so far be denoted by w.

August, 2005 Department of Computer Science, University of Ioannina Ioannina - GREECE 14 Expectation values The expectation value of the number of times the i th minimum is found, at the time when the w th minimum is recovered for the first time, is recursively given by: An estimation that may be used is:

August, 2005 Department of Computer Science, University of Ioannina Ioannina - GREECE 15 Keep trying … Suppose that after having found w minima, there is a number (say K) of consecutive trials without any success, i.e. without finding a new minimum. The expected number of times the i th minimum is found at that moment is given recursively by:

August, 2005 Department of Computer Science, University of Ioannina Ioannina - GREECE 16 The Observables’ criterion The quantity: Tends asymptotically to zero. Hence, STOP if:

August, 2005 Department of Computer Science, University of Ioannina Ioannina - GREECE 17 “Expected Minimizers” Rule  Based on estimating the number of local minima inside the domain of interest.  The estimation is improving as the algorithm proceeds.  The key quantity is the probability that l minima are found after m trials.  Calculated recursively.

August, 2005 Department of Computer Science, University of Ioannina Ioannina - GREECE 18 Probabilities If stands for the probability to recover in a single trial, then the probability of finding l minima in m trials is given by: Probability that a new minimum is found other than Probability that one of the first l minima is found again. Note that:

August, 2005 Department of Computer Science, University of Ioannina Ioannina - GREECE 19 Expected values The expected value for the number of minima, estimated after m trials is given by: The corresponding variance is given by: We use the estimate: STOP if : The RULE

August, 2005 Department of Computer Science, University of Ioannina Ioannina - GREECE 20 Other rules Uncovered fraction of space: Zieliński (1981) STOP if: Estimated number of minima: Boender & Rinnooy Kan (1987) STOP if: Probability all minima are found: Boender & Romeijn (1995) STOP if:

August, 2005 Department of Computer Science, University of Ioannina Ioannina - GREECE 21 Uncovered fraction Estimated # of minima Double BoxObservablesExpected # of minima MULTISTART

August, 2005 Department of Computer Science, University of Ioannina Ioannina - GREECE 22 Uncovered fraction Estimated # of minima Double BoxObservablesExpected # of minima TMLSL

August, 2005 Department of Computer Science, University of Ioannina Ioannina - GREECE 23 Conclusions  The new rules improve the performance at least for problems in our benchmark suite.  Proper choice of the parameter p, for different methods is important.  Remains to be seen if performance is also boosted in other practical applications.