1. Global Optimization: For Some Problems, There's HOPE Daniel M. Dunlavy Sandia National Laboratories, Albuquerque, NM, USA Dianne P. O’Leary Dept. of Computer Science and UMIACS University of Maryland, College Park, MD, USA Copper Mountain Conference on Iterative Methods April 3, 2006 SAND2006-2001C Sandia is a multiprogram laboratory operated by Sandia Corporation, a Lockheed Martin Company, for the United States Department of Energy’s National Nuclear Security Administration under contract DE-AC04-94AL85000.

2. Outline Unconstrained Minimization Problem Homotopy Optimization Methods Numerical Experiments Protein Structure Prediction Problem Numerical Experiments Concluding Remarks

3. Problem Solve the unconstrained minimization problem Function Characteristics –Solution exists –Smooth ( ) –Multimodal, deep local minima –Good starting points unavailable

4. Some Related Methods Stochastic search methods –Random perturbations Simulated annealing –Perturbations, acceptance criterion, schedule Evolutionary algorithms –Ensembles/populations Smoothing methods –Deformation of function being minimized Homotopy/continuation methods –Nonlinear equations,

5. Outline Unconstrained Minimization Problem Homotopy Optimization Methods Numerical Experiments Protein Structure Prediction Problem Numerical Experiments Concluding Remarks

6. Homotopy Optimization Method (HOM) Goal –Minimize complicated nonlinear target function Steps to solution –Easy template function:, is a known local min. –Define a continuous homotopy function: –Produce sequence of minimizers of w.r.t. starting at and ending at

7. Illustration of HOM

8. Homotopy Optimization using Perturbations & Ensembles (HOPE) Improvements over HOM –Produce ensembles of local minimizers of by perturbing intermediate results –Increase likelihood of predicting a global minimizer Algorithmic considerations –Maximum ensemble size –Determining ensemble members

9. Illustration of HOPE Maximum ensemble size = 2

10. Considerations using HOPE Template function –Known solution, relation to target function Homotopy function –Continuous, but smoothness may help Parameters –Maximum ensemble size –Perturbation function/amount of perturbation –Number of perturbed versions produced –Number of steps taken in Metric for choosing ensemble members

11. Convergence of HOPE HOPE parameterized as existing methods –Probability-one homotopy methods for NLPs –Stochastic search methods Pure Adaptive Search/Improving Hit-and-Run –Simulated Annealing HOPE converges in probability –Smooth homotopy: –Closed form for probability –Conditions for convergence with probability one Extension to general homotopy maps –One spatial dimension, –Basins of attraction must be known

12. Outline Unconstrained Minimization Problem Homotopy Optimization Methods Numerical Experiments Protein Structure Prediction Problem Numerical Experiments Concluding Remarks

13. Numerical Experiments Test Problems –N-modal sine function: –Moré, Garbow, Hillstrom test functions 5 functions where local methods failed –Pintér test function Random multimodal functions, unique minimizer known Standard homotopy functions –Convex, probability-one Comparisons with local and stochastic search –More computation using HOM/HOPE –Better results and ensembles of local minimizers

14. Outline Unconstrained Minimization Problem Homotopy Optimization Methods Numerical Experiments Protein Structure Prediction Problem Numerical Experiments Concluding Remarks

15. Protein Structure Prediction AlaArgAspAsnArg Protein StructureAmino Acid Sequence CC CN R H OH Given the amino acid sequence of a protein (1D), is it possible to predict its native structure (3D)?

16. Protein Structure Prediction: Computational Methods Molecular dynamics –Langevin dynamics approximated using stochastic differential equation Bioinformatics –Sequence/structure matches to experimentally determined native structures Comparative modeling (threading, homology modeling) Energy Minimization –Find lowest energy conformation Native structure [Anfinsen, 1973]

17. Some Existing Energy Minimization Methods Local methods –Truncated Newton and quasi-Newton methods Memory efficient (second derivatives not stored) Global methods –Stochastic search, simulated annealing, evolutionary algorithms, and smoothing methods –Other methods Convex global underestimation Stochastic tunneling Packet annealing Derivative-free pattern search

18. Energy Minimization using HOM Goal –Minimize energy function of target protein Steps to solution –Energy of template protein: –Define a homotopy function: Deforms template protein into target protein –Produce sequence of minimizers of starting at and ending at

19. Energy Minimization using HOPE Extensions of HOM –Perturbations Specific to protein structure Bond length, bond angle, and particle perturbations –Ensembles Ensembles chosen using homotopy function value Benefits over existing minimization methods –Take advantage of sequence-related structural properties of template and target proteins

20. Outline Problem and Existing Methods Homotopy Optimization Methods Numerical Experiments Protein Structure Prediction Problem Numerical Experiments Concluding Remarks

21. Backbone Model: Particle Properties Backbone model – Single chain of particles with residue attributes – Particles model C  atoms in proteins Properties of particles – Hydrophobic, Hydrophilic, Neutral – Diverse hydrophobic-hydrophobic interactions [Veitshans, et al., 1996.]

22. Backbone Model: Energy Function

24. Backbone Model: Experiments 9 chains (22 particles) with known structure Loop Region HydrophobicHydrophilicNeutral ABCDEFGHIABCDEFGHI

25. Backbone Model: Experiments 62 template-target pairs –10 pairs had identical native structures Methods –HOM vs. Newton’s method w/trust region (N-TR) –HOPE vs. ensemble-based simulated annealing (SA) Different ensemble sizes (2,4,8,16) Averaged over 10 runs Perturbations where sequences differ Measuring success –Structural overlap function: Percentage of interparticle distances off by more than 20% of the average bond length ( ) –Root mean-squared deviation (RMSD)

26. Backbone Model: Results

27. Success of HOPE and SA with ensembles of size 16 for each template-target pair. The size of each circle represents the percentage of successful predictions over the 10 runs. SAHOPE

28. Outline Problem and Existing Methods Homotopy Optimization Methods Numerical Experiments Protein Structure Prediction Problem Numerical Experiments Concluding Remarks

29. Conclusion New homotopy optimization methods –HOM: sequence of minimizers vs. path tracing –HOPE: perturbations and ensembles –Convergence (cast as existing methods) Numerical Experiments –HOM/HOPE outperform several standard methods –Standard test functions Standard homotopy functions used –Protein structure prediction Problem-specific homotopy functions Take advantage of sequence-related protein properties

30. Future Directions Protein structure prediction –More realistic energy functions (AMBER, CHARMM) –Protein Data Bank (templates) –Different size chains for template/target HOPE for large-scale problems –Inherently parallelizable –Communication: initializing ensembles at each step HOPE for other optimization problems –Constrained problems HOPE for other applications

31. Thank You Daniel M. Dunlavy – HOPE dmdunla@sandia.gov Publications D.M. Dunlavy, D.P. O'Leary, D. Klimov, and D. Thirumalai HOPE: A Homotopy Optimization Method for Protein Structure Prediction J. Comput. Biol., 12(10):1275-1288. Dec. 2005 HOPE: A Homotopy Optimization Method for Protein Structure Prediction D. M. Dunlavy and D.P. O'Leary Homotopy Optimization Methods for Global Optimization Sandia National Laboratories, SAND2005-7495. Dec. 2005 Homotopy Optimization Methods for Global Optimization D.M. Dunlavy Homotopy Optimization Methods and Protein Structure Prediction Ph.D. Thesis, University of Maryland, College Park, Aug. 2005 Homotopy Optimization Methods and Protein Structure Prediction D.M. Dunlavy Global Optimization of a Simplified Protein Energy Model In preparation Global Optimization of a Simplified Protein Energy Model