Easy Optimization Problems, Relaxation, Local Processing for a small subset of variables

Changes in the energy and overlap

X-direction line search and “discrete derivatives”  For node, fix all other nodes at their current  Current overlap  Calculate => choose sign  Calculate sign => quadratic approx.  To effectively calculate the derivative which means:  Calculate and average: Line search Discrete derivative

Different types of relaxation  Variable by variable relaxation – strict minimization  Changing a small subset of variables simultaneously – Window strict minimization relaxation  Stochastic relaxation – may increase the energy – should be followed by strict minimization

Window strict unconstrained minimization  Discrete (combinatorial) case : Permutations of small subsets P=2, placement

1D data base The nodes: A permutation  5 3   (1)= 7,  (2)=5,  (3)=2 … To find a consecutive subset of nodes in the current permutation, we need the inverse of  -1   -1 (1)= 5,  -1 (2)= 3,  -1 (3)= 9 … In 2D we have to insert a grid and store the list of nodes within each square

Window strict unconstrained minimization  Discrete (combinatorial) case : Permutations of small subsets P=2, placement Problem: very small number of variables!  Quadratic case : P=2

Window relaxation for P=2 unconstrained version  Minimize  Pick a window of variables, fix all variables at  Find a correction  to so as to minimize  Quadratic functional in many variables – easy to solve!

Updating the window variables  For each i in the window W insert the correction: x i new =x i +  i  Sort the x i new s and rearrange the window accordingly  To improve the result obtained by the inner changes apply node-by-node relaxation on W and on its boundary  At the and compare the “old” energy with the “new” energy and accept / reject  Revision process: try a “big” change, improve it by local minimization, choose

Window relaxation for P=2 constrained version  To prevent nodes from collapsing on each other  To express the aim of having an approximate permutation of add 2 constraints:

Exc#4: Permutation’s invariants 1)Prove that are invariant under permutation. 2) Is it also true for m=3?

Window relaxation for P=2 constrained version  To prevent nodes from collapsing on each other  To express the aim of having an approximate permutation of add 2 constraints:  Minimization with equality constraints  Lagrange multipliers

Lagrange multipliers  Goal: Transform a constrained optimization problem with n variables and m equality constraints to an unconstrained optimization problem with n+m variables. The new m variables are called the Lagrange multipliers  Geometry explanation

2 constraints in 3D

The optimal ellipsoid is tangent to the constraints curve

Lagrange multipliers  Goal: Transform a constrained optimization problem with n variables and m equality constraints to an unconstrained optimization problem with n+m variables. The new m variables are called the Lagrange multipliers  Geometry explanation  Construct an augmented functional – the Lagrangian

The Lagrangian Given E(x) subject to m equality constraints: h k (x)=0, k=1,…,m, construct the Lagrangian L(x, ) = E(x) +  k  k  h k (x) and solve the system of n+m equations  The value of  is meaningful The constraints!

The Lagrangian: an example  Minimize E(x,y)=x+y  Subject to h(x,y)=x 2 +y 2 -2  The Lagrangian: L(x,y, =E(x,y)+ (x 2 +y 2 -2  The constraint! The co-linearity of the gradients

Window relaxation for 1D ordering constrained/unconstrained version  Minimize  Pick a window of variables, fix all variables at  Find a correction  to  Update the window’s variables, restore volume constraints and revise around the window  Switch to the next window chosen with overlap  Use a (small) sequence of variable size windows  For example use windows with 5,10,15,20,25 nodes

Easy to solve problems  Quadratic functional and linear constraints  Solve a linear system of equations  Quadratization of the functional: P=1, P>2

Quadratization for P=1 and P>2  Minimize ;  Given a current approximation  Minimize

Easy to solve problems  Quadratic functional and linear constraints  Solve a linear system of equations  Quadratization of the functional: P=1, P>2  Linearization of the constraints: P=2

Window relaxation for P=2 constrained version  To prevent nodes from collapsing on each other  To express the aim of having an approximate permutation of add 2 constraints:  The terms were neglected assuming they are small enough compared with other terms in the equation

Easy to solve problems  Quadratic functional and linear constraints  Solve a linear system of equations  Quadratization of the functional: P=1, P>2  Linearization of the constraints: P=2  Inequality constraints: active set method