1. Nizhni Novgorod State University Faculty of Computational Mathematics and Cybernetics Searching Globally-Optimal Decisions for Multidimensional Optimization Problems Sysoyev A. Software Department 2007

2. 2 Contents Optimization problem Global optimization technique Informational Statistical Approach Index method Parallelization Computing with arbitrary precision math Visualization Results

3. Optimization problem… Global optimization problem is stated as follows: Criteria and constraints are continuous and satisfy the Lipschitz condition 3

4. Optimization Problem… Given problem belongs to one class of time- consuming problems –Problem of integration –The solution of the system of nonlinear equations –Problem of multiextremal multidimensional problems –Problem of reconstruction of dependences, etc… 4

5. 5 Optimization problem Necessity of parallel computations Global optimization problems can be: –multidimensional –multiextremal –nonconvex and multiply-connected –time-consuming at function value computations. So these problems are highly computational intensive y*

6. 6 To solve optimization problems various iterative schemes can be applied An iteration: –Choosing a current point y n in the search domain –Calculating the problem functions at the point y n Iterations are terminated by the given time limitations or the accuracy Global Optimization Technique… ynyn

7. 7 Naive method – the scanning technique The number of iterations grows exponentially with increasing of problem dimension Example Global Optimization Technique…

8. 8 More efficient approach is to use iterative procedures which generate iteration points with nonuniform density in the search domain, viz.: –More dense in locality of function extreme –Less dense otherwise Among such techniques – Informational Statistical Approach Global Optimization Technique

9. 9 Informational Statistical Approach Lipschitz condition We propose the limited variation Δy of the argument y generates restricted variations Δ F i (y) In a wide set of problems arising in applications this condition is satisfied as a consequence of limitations on energy changes inherent to real systems to be modeled Such proposal shows a way to construct non-uniform grids to solve the problems

10. 10 Informational Statistical Approach Successive scheme to generate nodes of the grid To utilize efficiently any information obtained at the time of calculation we have to generate nodes of the grid in a successive way where the values Z l,1  l  k, being arguments of the decision function G k, are values of the vector-function F(y), computed at the nodes y l,1  l  k, of the grid Y k. Such approach poses a problem of accumulating all computed information as the data base

11. 11 Informational Statistical Approach Dimension reduction A possible way to cut down the complexity of formulating the decision function G k is to reduce multidimensional problems It can be provided by using the continuous single-valued correspondence of Peano type curve The Lipschitz condition is transformed to the Hölder condition 0 1 …

12. 12 Successive scheme of function calculations: - - if, then - if, then… … - if, then - Result – the value of the last calculated function and its number (index) Informational Statistical Approach Index scheme

13. 13 Informational Statistical Approach Scheme of iterations Choice of a current iteration point: –Computing the interval characteristics –Choosing the interval with maximum characteristic –Selecting the current point x0x0 x1x1 x2x2 xkxk x k+1 x n-2 xnxn R0R0 R1R1 R n-1 RkRk

14. 14 Index method A weak spot Close points in R N – can be not close on [0, 1] 0 1

15. 15 Index method Multiple mappings… 0 1 … 0 1 … 0 1 … 0 1 … 0 1 … A possible way for overcoming this disadvantage is using the set of joint space-filling curves

16. 16 Index method Multiple mappings 01 2 LL+1 … L-1 unfeasible point feasible point Multiple mappings produces the corresponding set of univariate global optimization problems

17. 17 Parallelization Approach Such information unity gives a possibility to solve the function set concurrently Process 0 0 1 i-1i L+1 L Process i Process L 1) 2) 3) queue … … mapping function calculations

18. 18 Parallelization Scheme of parallel computations… Choosing a point of a new iteration

19. 19 Parallelization Scheme of parallel computations… Each process informs all other ones

20. 20 Parallelization Scheme of parallel computations Each process sends a message including the function value obtained to all other processes

21. 21 Computing with arbitrary precision math Problem –To get the accuracy ε = (½) m in R N the accuracy (½) m·N in [0, 1] should be applied –If the C++ type double is used the maximum possible accuracy is limited by the condition Nm < 52 (if m = 10 than the problem dimension have to N <= 5) Decision –To use arbitrary precision numbers to represent points in [0, 1]

22. 22 Visualization…

23. 23 Visualization…

24. 24 Visualization

25. 25 Results… Coordinates of optimizer: x1 = 0.942, x2 = 0.946 Proc. х mappingIterations*Speed up 1 x 16 2000 - 2 х 16 1015 1.97 4 х 16 519 3.85 8 х 16 270 7.41 16 х 16 147 13.61 * - Iterations means maximum of the number of feasible points calculated by processors

26. 26 Results… Coordinates of optimizer: x1 = 0.942, x2 = 0.946 Proc. х mappingIterations*Speed up 1 x 32 2000 - 2 х 32 1081 1.85 4 х 32 577 3.46 8 х 32 282 7.09 16 х 32 172 11.63 20 х 20 118 16.95 * - Iterations means maximum of the number of feasible points calculated by processors

27. 27 Results… Rastrigin’s function Proc. х NIterations*Speed up 1 x 8 1000 - 8 х 8 147 6.80 16 х 8 69 14.49 1 х 10 1000 - 8 х 10 150 6.67 16 х 10 75 13.33 * - Iterations means maximum of the number of feasible points calculated by processors

28. 28 Results… Rastrigin’s function

29. 29 Results… Applied problems solved: –The problem «Parameter identification in a model of economics» (jointly with The Computing Center of RAS, Moscow) –The problem «Optimization of parameters of a measurement system» (jointly with The Research Institute «Burevestnik», Nizhni Novgorod) –The optimization system developed is applied for solving applied optimization problems in the Delft University, Netherland

30. 30 Results «Parameter identification in a model of economics» –12 parameters in search domain –2 functional constraints –The task was solved on MVS-1000 by scanning technique with 8 points on each dimension. 500 processors was used. Solving time was about 5 hours –The task was solved on single PC by Global Expert system with 16 points on each dimension. Solving time was about 2 minutes.

31. 31 Questions, Remarks, Something to add…