Multifidelity Optimization Using Asynchronous Parallel Pattern Search and Space Mapping Techniques Genetha Gray*, Joe Castro i, Patty Hough*, and Tony Giunta d *Computational Sciences & Mathematics Research, Sandia National Labs, Livermore, CA i Computational Sciences and d Validation & Uncertainty Quantification Processes, Sandia National Labs, Albuquerque, NM Sandia National Laboratories is a multiprogram laboratory operated by Sandia Corporation, a Lockheed Martin Company, for the United States Department of Energy under contract DE-AC04-94AL Purpose of Multifidelity Optimization Outer Loop Optimizer: Asynchronous Parallel Pattern Search (APPSPACK) MFO Scheme Space Mapping Representation Ongoing and Future Work References Polynomial Example The Inner Loop as an Oracle Result: Gamma ~ O (1), Starting Point = (-2.0,-2.0)  Study spaces defined using different constraints.  Implement a generic oracle in APPSPACK.  Include a space mapping that does not require domain spaces to be defined by the same numbers of parameters.  Apply our multifidelity optimization schemes to some real world problems:  Earth penetrator analysis  Groundwater problems including well field design & hydraulic capture  Circuit system design  Bakr, Bandler, Madsen, and Sondergard, An Introduction to Space Mapping Technique, Optimization & Engineering, 2: ,  Gray and Kolda, APPSPACK 4.0: Asynchronous Parallel Pattern Search for Derivative-Free Optimization, Technical Report SAND , Sandia National Laboratories, Livermore, CA, Aug (Submitted to ACM TOMS.)  Kolda, Revisiting Asynchronous Parallel Pattern Search for Nonlinear Optimization, Technical Report SAND , Sandia National Laboratories, Livermore, CA, Feb (In revision for SIAM J. Opt.)  Kolda, Lewis, and Torczon, Optimization by Direct Search: New Perspectives on Some Classical and Modern Methods, SIAM Review, 45(3): , Consider a simple example with constant space mapping terms: Study space mapping sensitivities to various inputs –Number of high fidelity responses used in the mapping –Scaling of the mapping parameters (size of the offset between the low and high fidelity models) –Starting point Compare the optimum obtained and the number of high fidelity runs required to reach this optimum A Simple Example # responses (x 0,x 1 ) (-0.5,0.83) Objective Value 0.0 # Hi-Fi Calculations Speed Up Ratio 2(-0.50, 0.83)8.86e (-0.50, 0.83)5.13e (-0.52, 0.84)3.34e (-0.48, 0.81)1.03e Apps only(-0.50, 0.85)4.00e-458- View of High Fidelity Design Space Log Plot of Results 1 2 Hi-Fi Model (-0.76,2.0)(-0.8,-1.2) 1 2  The numbered white boxes show approximately where the inner loop was called.  The point in red brackets is the APPSPACK best point before the inner loop call.  The point in green was obtained by the inner loop. (-0.56,1.6) (-0.61,1.25) When the number of response points is 8, there are two calls to the inner loop …  Direct search method  Takes advantage of parallel platform to reduce computational time  Does not assume time needed to evaluate objective function is constant  Global convergence to a stationary point under mild conditions  Does not assume homogeneous processors  If the inner loop is viewed as an oracle, the only significant algorithmic change to APPS is the addition of the oracle.  Analytically, there are no restrictions on how an oracle can choose points.  Oracle points are used in addition to the points defined by the search pattern  The convergence of APPSPACK is not adversely affected.  A convergence proof for the APPS algorithm exists so no new convergence proof for our MFO scheme is needed.  Future work may include investigating any improvement to the convergence of APPS.  In many optimization applications, the objective function f(x) is expensive to calculate and derivatives may be inaccurate if they exist at all.  Many of these applications have a high fidelity (or true) model and a low fidelity (or surrogate) model which simplifies the high fidelity model in some way.  An MFO approach optimizes an inexpensive, low fidelity model while making periodic corrections using the expensive, high fidelity model.  The relationship between the models can be exploited in an algorithm by applying space mapping techniques. Outer Loop Inner Loop Low Fidelity Model Optimization  x H    High Fidelity Mode Optimization via APPSPACK Space Mapping Via Least Squares Calculation  x H,f(x H ) x H trial MFO ALGORITHM 1.Start Outer Loop (APPSPACK)  Evaluate N high fidelity response points  Produce “table” of data with x H and f H (x H ) 2.Start Inner Loop  Take data from APPSPACK table  Run LS optimization algorithm  At each iteration, evaluate N low-fidelity responses  At completion obtain , ,  for our space mapping of  (x H )  +   Optimize low fidelity model within space mapped high fidelity space. In other words, minimize f L (  (x H )  +  ) with respect to x H to obtain x H * 3.Return x H * to APPPSPACK to determine if a new best point has been found.