Global Optimization: For Some Problems, There's HOPE Daniel M. Dunlavy Sandia National Laboratories, Albuquerque, NM, USA Dianne P. O’Leary Dept. of Computer Science and UMIACS University of Maryland, College Park, MD, USA Copper Mountain Conference on Iterative Methods April 3, 2006 SAND C Sandia is a multiprogram laboratory operated by Sandia Corporation, a Lockheed Martin Company, for the United States Department of Energy’s National Nuclear Security Administration under contract DE-AC04-94AL85000.

Outline Unconstrained Minimization Problem Homotopy Optimization Methods Numerical Experiments Protein Structure Prediction Problem Numerical Experiments Concluding Remarks

Problem Solve the unconstrained minimization problem Function Characteristics –Solution exists –Smooth ( ) –Multimodal, deep local minima –Good starting points unavailable

Some Related Methods Stochastic search methods –Random perturbations Simulated annealing –Perturbations, acceptance criterion, schedule Evolutionary algorithms –Ensembles/populations Smoothing methods –Deformation of function being minimized Homotopy/continuation methods –Nonlinear equations,

Outline Unconstrained Minimization Problem Homotopy Optimization Methods Numerical Experiments Protein Structure Prediction Problem Numerical Experiments Concluding Remarks

Homotopy Optimization Method (HOM) Goal –Minimize complicated nonlinear target function Steps to solution –Easy template function:, is a known local min. –Define a continuous homotopy function: –Produce sequence of minimizers of w.r.t. starting at and ending at

Illustration of HOM

Homotopy Optimization using Perturbations & Ensembles (HOPE) Improvements over HOM –Produce ensembles of local minimizers of by perturbing intermediate results –Increase likelihood of predicting a global minimizer Algorithmic considerations –Maximum ensemble size –Determining ensemble members

Illustration of HOPE Maximum ensemble size = 2

Considerations using HOPE Template function –Known solution, relation to target function Homotopy function –Continuous, but smoothness may help Parameters –Maximum ensemble size –Perturbation function/amount of perturbation –Number of perturbed versions produced –Number of steps taken in Metric for choosing ensemble members

Convergence of HOPE HOPE parameterized as existing methods –Probability-one homotopy methods for NLPs –Stochastic search methods Pure Adaptive Search/Improving Hit-and-Run –Simulated Annealing HOPE converges in probability –Smooth homotopy: –Closed form for probability –Conditions for convergence with probability one Extension to general homotopy maps –One spatial dimension, –Basins of attraction must be known

Outline Unconstrained Minimization Problem Homotopy Optimization Methods Numerical Experiments Protein Structure Prediction Problem Numerical Experiments Concluding Remarks

Numerical Experiments Test Problems –N-modal sine function: –Moré, Garbow, Hillstrom test functions 5 functions where local methods failed –Pintér test function Random multimodal functions, unique minimizer known Standard homotopy functions –Convex, probability-one Comparisons with local and stochastic search –More computation using HOM/HOPE –Better results and ensembles of local minimizers

Outline Unconstrained Minimization Problem Homotopy Optimization Methods Numerical Experiments Protein Structure Prediction Problem Numerical Experiments Concluding Remarks

Protein Structure Prediction AlaArgAspAsnArg Protein StructureAmino Acid Sequence CC CN R H OH Given the amino acid sequence of a protein (1D), is it possible to predict its native structure (3D)?

Protein Structure Prediction: Computational Methods Molecular dynamics –Langevin dynamics approximated using stochastic differential equation Bioinformatics –Sequence/structure matches to experimentally determined native structures Comparative modeling (threading, homology modeling) Energy Minimization –Find lowest energy conformation Native structure [Anfinsen, 1973]

Some Existing Energy Minimization Methods Local methods –Truncated Newton and quasi-Newton methods Memory efficient (second derivatives not stored) Global methods –Stochastic search, simulated annealing, evolutionary algorithms, and smoothing methods –Other methods Convex global underestimation Stochastic tunneling Packet annealing Derivative-free pattern search

Energy Minimization using HOM Goal –Minimize energy function of target protein Steps to solution –Energy of template protein: –Define a homotopy function: Deforms template protein into target protein –Produce sequence of minimizers of starting at and ending at

Energy Minimization using HOPE Extensions of HOM –Perturbations Specific to protein structure Bond length, bond angle, and particle perturbations –Ensembles Ensembles chosen using homotopy function value Benefits over existing minimization methods –Take advantage of sequence-related structural properties of template and target proteins

Outline Problem and Existing Methods Homotopy Optimization Methods Numerical Experiments Protein Structure Prediction Problem Numerical Experiments Concluding Remarks

Backbone Model: Particle Properties Backbone model – Single chain of particles with residue attributes – Particles model C  atoms in proteins Properties of particles – Hydrophobic, Hydrophilic, Neutral – Diverse hydrophobic-hydrophobic interactions [Veitshans, et al., 1996.]

Backbone Model: Energy Function

Backbone Model: Experiments 9 chains (22 particles) with known structure Loop Region HydrophobicHydrophilicNeutral ABCDEFGHIABCDEFGHI

Backbone Model: Experiments 62 template-target pairs –10 pairs had identical native structures Methods –HOM vs. Newton’s method w/trust region (N-TR) –HOPE vs. ensemble-based simulated annealing (SA) Different ensemble sizes (2,4,8,16) Averaged over 10 runs Perturbations where sequences differ Measuring success –Structural overlap function: Percentage of interparticle distances off by more than 20% of the average bond length ( ) –Root mean-squared deviation (RMSD)

Backbone Model: Results

Success of HOPE and SA with ensembles of size 16 for each template-target pair. The size of each circle represents the percentage of successful predictions over the 10 runs. SAHOPE

Outline Problem and Existing Methods Homotopy Optimization Methods Numerical Experiments Protein Structure Prediction Problem Numerical Experiments Concluding Remarks

Conclusion New homotopy optimization methods –HOM: sequence of minimizers vs. path tracing –HOPE: perturbations and ensembles –Convergence (cast as existing methods) Numerical Experiments –HOM/HOPE outperform several standard methods –Standard test functions Standard homotopy functions used –Protein structure prediction Problem-specific homotopy functions Take advantage of sequence-related protein properties

Future Directions Protein structure prediction –More realistic energy functions (AMBER, CHARMM) –Protein Data Bank (templates) –Different size chains for template/target HOPE for large-scale problems –Inherently parallelizable –Communication: initializing ensembles at each step HOPE for other optimization problems –Constrained problems HOPE for other applications

Thank You Daniel M. Dunlavy – HOPE Publications D.M. Dunlavy, D.P. O'Leary, D. Klimov, and D. Thirumalai HOPE: A Homotopy Optimization Method for Protein Structure Prediction J. Comput. Biol., 12(10): Dec HOPE: A Homotopy Optimization Method for Protein Structure Prediction D. M. Dunlavy and D.P. O'Leary Homotopy Optimization Methods for Global Optimization Sandia National Laboratories, SAND Dec Homotopy Optimization Methods for Global Optimization D.M. Dunlavy Homotopy Optimization Methods and Protein Structure Prediction Ph.D. Thesis, University of Maryland, College Park, Aug Homotopy Optimization Methods and Protein Structure Prediction D.M. Dunlavy Global Optimization of a Simplified Protein Energy Model In preparation Global Optimization of a Simplified Protein Energy Model