1 Valery I. Zorkaltsev, Professor, Head of Laboratory, Energy Systems Institute Siberian Branch of the Russian Academy of Sciences International conference “Optimization and applications" Montenegrio 2009 г. SYMMETRIC DUALITY IN OPTIMIZATION AND IT’S APPLICATIONS

2 For wide class of optimization problems they use special constructions called Dual optimization problems:, where primal optimization problem; dual optimization problem; transition rule (often polysemantic). Definition of symmetric duality

3 For dual problem one can specify problem dual to it Symmetric duality is event, when dual problem to dual problem coincides with primal problem

4 Applications of dual problems: to prove optimality of obtained solutions; for justification of optimization algorithms; in solution interpretation; for making optimization algorithms; for researching and solving many complicated problems of operation research, including Nash equilibrium finding.

5 Lecture plan 1. Theory of symmetric duality in optimization: –Lagrangian multipliers; –Theorems of alternative systems of linear inequalities; –Legendre-Fenchel conjugate functions and their extensions. 2. Application of symmetric duality in optimization algorithms and regularization

6 Lecture plan (continuation) 3. Application in models: –load-flow models (electric circuits, hydraulic circuits, nonlinear transportation problems); –models of thermodynamic equilibrium and geometric programming; –economic equilibrium models.

7 1. Lagrange multipliers of constraints Primal problem: (1) Lagrange problem: where Lagrange multipliers, which satisfy conditions (1). Modified Lagrange problem (Sh. Churkveidze):

8 2. Theory of alternative systems of linear inequalities Any system of linear inequalities can be confronted with an alternative system of linear inequalities S by formal rules so that proposition is right: One and only one system of two is consistent: S or S *. Moreover, backward transformation takes place: and. That is alternative systems S and S * are symmetric.

9 Three examples of theorems of alternative systems of linear inequalities It is assigned: А – matrix, b – vector in R m. Sought vectors –,. Remark. System of linear equations can be considered as special case of system of linear inequalities. The converse proposition is not correct.

10 1. Fredgolm’s theorem (about alternative systems of linear inequalities) Either there is or there is

11 2. Farkas’ theorem Either the following system possesses a solution or the following system is solvable

12 3. Gail’s theorem Either there is such, that or there is vector such, that

13 Applications of alternative systems of linear inequalities theory 1.Identification of system of linear inequalities incompatibility –If a vector from the solution set of an alternative system S* will be obtained during the process of searching the solution of system S, then absence of the solution of initial system S will be proved. We have practical and effective (as computation has shown) method for identification of problem constraints inconsistency.

14 2. For determination of redundant constraints, exclusion of which doesn’t change the solution set, including situations in algorithms – Gomory or Kelly cuts; – Fourier-Chernikov convolutions for description of systems of linear inequalities solutions. 3. For identification of solutions of systems of linear inequalities with minimal set of active constraints – relative to interior points of systems of linear inequalities solution set.

15 4. For creation of new algorithms for solving systems of linear and on the basis of this nonlinear inequalities («Alternative approach», which is developed by U. Evtushenko, A. Golicov). 5. All theory of linear optimization duality is contained in theorems of alternative systems of linear inequalities. Duality of linear optimization is the basis for wide class of nonlinear problems.

16 Search of solutions and identification of inconsistence of system of double-sided linear inequalities The more restricted class of problems is considered the more interesting results about characteristics of this class of problems can be obtained Initial system: find satisfying the following conditions Alternative system of one inequality: find, such that where Here for vectors, have components:

17 Comparison of variants of interior point methods for problems of permissible regimes of electric power systems Algo- rithm Number of iterations for problems inconsistentconsistent 6*740*802*719*19201*201 A B C D11558 E A, B – primal algorithms C, D – dual algorithms Е – primal-dual algorithm

18 Mutually dual problems of linear programming Let be sets of optimal solutions of problems (P), (P*). Let’s introduce sets of recession directions for this problems :, According to Farkas and Geil theorems pairs of sets and are alternative. Symmetric duality takes place for LP problems:

19 Theorem of duality for LP Four events are possible for problems (Р), ( ) : 1. If, then,,,. 2. If,. Тогда,,,. 3. If,. Тогда,,,.

20 Theorem of duality for LP (continuation) 4. If,. Then,,,. For any, There is, such that In this and only this case.

21 Equivalent representations of LP problem in the form of optimization problems 1. Primal problem 2. Dual problem 3. Self-dual problem 4. Symmetric problem (problem of complementa- rity)

22 Reresentation of a linear programming problem as a system of linear inequalities It allows to consider problems of linear programming as a special case of systems of linear inequalities.

23 3. Conjugate functions for 1. Functions is Legendre conjugate of each other if where That is

24 3. Conjugate functions 2. Functions is Fenchel conjugate of each other, if and

25 3. Generalization of conjugate functions of Legendre-Fenchel 3. Functions is conjugate of each other, if where symmetric positive defined matrix. Following functions are mutually inverse Following inequality is held

26 Symmetric duality 1. Primal problem (S) (S) (1) 2. Dual problem (S * ) (S * ) (2) Note: problems (S) and (S * ) have different structure of variables; dual to dual problem coincide with primal problem.

27 Symmetric duality (continuation) 3. Self-dual problem: subject to (1), (2) 4. Symmetric problem: subject to (1), (2)

28 Symmetric duality. Equivalent system of equalities and inequalities Constraint (3) can be substituted with еquality

29 Examples I. Symmetric duality for problems of quadratic programming where positive definite matrix Mutually dual problems

30 Examples II. Especially important case of separable functions One form of writing the equivalent system of equalities and inequalities

31 Theorem (for separable ) Let f j be continious increasing functions, then (S) is a problem of minimization of strictly convex function with linear constraints, (S * ), (SD) are problems of minimization of convex function with linear constraints. If, at the same time, and a system Ax=b, is consistent, then problems (S), (S * ), (SD), (SS), (L) have coincident and unique (relatively to vectors x, y) solutions. Vector u is unique if rank A=m.

32 Applications, separable case 1. Regularization of linear programming problems: having small Primal problem: regularization by Tihonov Dual problem: search of pseudosolution of dual problem of linear programming

33 Applications (with and self-conjugated functions of the kind ) 2. «Alternative» way of searching for normal solutions for system of inequalities with n variables This is equivalent to problem with m variables Such approach is preferable when

34 Applications 3. Load-flow models (nonlinear transportation problems, electric circuits, hydraulic circuits including heat, water and gas delivery problems) indices of nodes, indices of arcs, incedence matrix, vector of volumes of delivery in system and out of system,

35 Load-flow models vector of pressure gains (or electro- motive forces, or conveyance tariffs) on arcs, vector of flows on arcs, vector of pressures (tensions, prices) in nodes, vector of pressure losses (tension losses, price rises) on arcs.

36 Load-flow models flow balance in nodes (first Kirchhoff law), balances of pressures on arcs, interrelations of pressure loss and flow on arcs. For example, Ohm law, Darcy law.

37 Results for hydraulic circuits obtained using the theory of symmetric duality 1.Conditions for existence and uniqueness of classical load-flow model solution are clarified. 2.Possibilities for choosing the form of mathematical models representation are expanded. 3.Foundations for constructing and theoretical justification of algorithms for solving load- flow problems are obtained.

38 Results for hydraulic circuits obtained using the theory of symmetric duality 4.Theoretical research is held (including clarification of conditions for existence and uniqueness), algorithms for solving non- classical load-flow problems are developed, where some components of vectors x, y, u, b and c may be fixed аnd other components of these vectors should be found.

39 Transport model with piecewise defined nonlinear costs Model is applied in analysis of operation of natural gas and oil delivery systems to find and eliminate bottlenecks in proper time. Let be flow through the arc j, − costs coefficient for the arc j,, − nonlinear function. For each arc costs function will be

40 where for costs flow Normal regime Extremal regime

41 Primal optimization problem b i – volume of delivery into the net at source node (if ) or out of the net at consumer node (if ), h i – penalty for incomplete delivery in node i, I src – set of numbers of source nodes, I cons – set of numbers of consumer nodes.

42 Economic interpretation 0 – profit (surplus) of transport company on arc j – transportation costs on arc j – revenue from transportation on arc j Fig. 1. Plot of marginal costs

43 Calculation experiments results Table 1. Results of computations Results of calculations for method of interior points on number of example networks (Number of nodes, number of arcs) Amount of iterations of interior points method Time of compu- tation, sec Achieved accuracy of equality constraints Achieved accuracy of optimality conditions (25, 30) * (50, 67) * (75, 109) * (100, 116) * (200, 240) *

44 Diagrams for computation results Number of iterations Amount of variables Time of com- putation, sec Amount of variables Results of calculations is shown on two diagrams

45 Problem of finding bottlenecks in natural gas delivery network in order to obtain system reliability Two examples were computed for real networks: –Aggregated network for natural gas delivery system (21 nodes, 28 arcs) –Detailed network for the same system (337 nodes, 589 arcs) Two aims of computation for each example: –1) to determine nodes with low supply and arcs with utilized capacity when only normal regime is allowed –2) to determine abilities to increase supply of nodes with low supply and find arcs switched to extremal regime when extremal regime is allowed

46 Aggregated network. Only normal regime is allowed Aggregated network. Extremal regime is allowed Linear load-flow Nonlinear load-flow

47 Detailed network for natural gas delivery system Number of nodes: 337 Number of arcs: 589 Amount of iterations of interior points method: 82 Time of calculation: sec

48 Final word I’d like to give thank to people who helped me make this report: –Perjabinsky Sergey, –Medvezhonkov Dmitry. Thank you for your attention!