Easy Optimization Problems, Relaxation, Local Processing for a single variable

Multiscale solvers Coarsening: create a hierarchy of problems graphs, equations, systems of particles, etc.

Original system 1 st coarsening 2 nd coarsening 3 rd coarsening

Multiscale solvers Coarsening: create a hierarchy of problems graphs, equations, systems of particles, etc. Solve the coarsest level

Coarsest level solution

Multiscale solvers Coarsening: create a hierarchy of problems graphs, equations, systems of particles, etc. Solve the coarsest level Uncoarsening:  Initialize the solution on a finer level from the coarser level by interpolation  Improve the initial solution by local processing

Local processing Main assumption The solution of the larger scales has been obtained by the coarser levels  At each level apply only local changes  Since done iteratively, need not solve to the optimum, just approach it

Variable by variable strict unconstrained minimization  Discrete (combinatorial) case : Ising model

2D Ising spins  Minimize  Periodic boundary condition  Initialize randomly: with probability.5

Exc#1: 2D Ising spins exercise  Minimize  Periodic boundary condition  Initialize randomly: with probability.5 1.Go over the grid in lexicographic order, for each spin choose 1 or -1 whichever minimizes the energy (choose with probability ½ when the two possibilities have the same energy) until no changes are observed. 2. Repeat 3 times for each of the 4 possibilities of (h 1,h 2 ). 3. Is the global minimum achievable? 4. What local minima do you observe?

Variable by variable strict unconstrained minimization  Discrete (combinatorial) case : Ising model  Quadratic case : P=2

Necessary optimality conditions Let be a local minimum of and assume is continuously differentiable in some domain, then the 1 st order Necessary Condition is If in addition is twice continuously differentiable within, then the 2 nd order Necessary Condition is positive semidefinite

Sufficient optimality conditions Let be twice continuously differentiable in domain and let satisfy the conditions, positive definite then is a strict unconstrained local minimum of. If, in addition, is quadratic, the local minimum is also the global unique minimum.

Pointwise relaxation for P=2  Minimize  Pick a variable, fix all at  Minimize Quadratic functional in one variable – easy to solve!

Pointwise relaxation for P=2 (cont.)  Check the 2 nd derivative: => Unique minimum! Put at the weighted average location of its graph neighbors  Go over all variables in lexicographic order Problem: Does not preserve the volume demands! Reinforce volume demands at the end of each sweep

Variable by variable strict unconstrained minimization  Discrete (combinatorial) case : Ising model  Quadratic case : P=2  General functional : P=1, P>2

Exc#2: Pointwise relaxation for P=1  Minimize  Pick a variable, fix all at  Minimize  Find the optimal location for