1. Multifidelity Optimization Via Pattern Search and Space Mapping Genetha Gray Computational Sciences & Mathematics Research Sandia National Labs, Livermore, CA Sandia is a multiprogram laboratory operated by Sandia Corporation, a Lockheed Martin Company, for the United States Department of Energy’s National Nuclear Security Administration under contract DE-AC04-94AL85000.

2. Outline  Multifidelity Optimization  APPSPACK  Space Mapping  MFO scheme  Descriptive Example  Groundwater Remediation Example

3. Multifidelity Optimization (MFO)  The low fidelity model retains many of the properties of the high fidelity model but is simplified in some way Decreased physical resolution Decreased physical resolution Decreased FE mesh resolution Decreased FE mesh resolution Simplified physics Simplified physics  MFO optimizes an inexpensive, low fidelity model while making periodic corrections using the expensive, high fidelity model.  Works well when low-fidelity trends match high-fidelity trends. Low Fidelity 30,000 DOF High Fidelity 800,000 DOF Finite Element Models of the Same Component

4. Asynchronous Parallel Pattern Search (APPS)  Direct method → no derivatives required  Pattern of search directions drives search and determines new trial points for evaluation  Objective function can be an entirely separate program  Achieves parallelism by assigning function evaluations to different processors  Freely available software under the GNU public license (APPSPACK)

5. Synchronous Pattern Search Inherently (or embarrassingly) parallel, but processor load should be considered.

6. Processor Load Balance Considerations  The number of trial points can vary at each iteration. Cached function values Cached function values Search patterns change Search patterns change Constraints (infeasible trial points are not evaluated) Constraints (infeasible trial points are not evaluated)  Evaluation times can vary for each trial point. Different processor characteristics Different processor characteristics Effect of input on function Effect of input on function Function evaluation faults Function evaluation faults MFO: different function models with different evaluation times!! MFO: different function models with different evaluation times!!

7. APPSPACK Example Workers Waiting

8. APPSPACK Example Workers Waiting b c d e

9. APPSPACK Example Workers Waiting c d

10. APPSPACK Example Workers Waiting f c d g

11. APPSPACK Example Workers Waiting c d

12. APPSPACK Example Workers Waiting h c d i j,k

13. APPSPACK Example Workers Waiting i

14. APPSPACK Example Workers Waiting l m n i o

15. APPSPACK Example Workers Waiting i

16. APPSPACK Example Workers Waiting p q r i s

17. APPSPACK Example Workers Waiting s Note: Cannot Prune on Unsuccessful Iteration

18. APPSPACK Example Workers Waiting s u t v

19. Space mapping* is a technique that maps the design space of a low fidelity model to the design space of high fidelity model such that both models result in approximately the same response. The parameters within x H need not match the parameters within x L Space Mapping*: A Conduit Between the Low and High Fidelity Model Design Spaces x – design variables R - response P - mapping xHxH RH(xH)RH(xH) high-fi model xLxL RL(xL)RL(xL) low-fi model *Developed by John Bandler, et. al. xHxH R L (P(x H )) mapped low-fi model P(x H ) xL=P(xH)xL=P(xH) R L (P(x H ))  R H (x H ) such that We’re using the mapping ?

20. Oracle  An oracle predicts points at which a decrease in the objective function might be observed.  Analytically, an oracle can choose points by any finite process.  Oracle points are used in addition to the points defined by the search pattern.  The MFO scheme employs an oracle framework to do a space mapping so that APPSPACK convergence is not adversely affected.  Future work may include investigating any convergence improvement.

21. The MFO Scheme: Combining APPSPACK and Space Mapping Outer Loop Inner Loop Low Fidelity Model Optimization  x H    High Fidelity Mode Optimization via APPSPACK Space Mapping Via Nonlinear Least Squares Calculation  multiple x H,f(x H ) x H trial

22. MFO Algorithm 1. 1. Start the Outer Loop (APPSPACK)   Evaluate N high fidelity response points   Produce x H, f H (x H ) pairs 2. 2. Start the Inner Loop   Take data pairs from APPSPACK   Run LS optimization   At each iteration, evaluate N low fidelity responses   At conclusion, obtain , ,  for space map  (x H )  +    Optimize low fidelity model within space mapped high fidelity space. In other words, minimize f L (  (x H )  +  ) with respect to x H to obtain x H *. 3. 3. Return x H * to APPSPACK to determine if a new best point has been found.

23. View of High Fidelity Design Space View of Unmapped Low Fidelity Design Space A Simple Example

24. MFO Results

25. 1 When the # response points is 8, there are two calls to inner loop. Approximate Inner Loop Call Locations within Hi-Fi Model (-0.76,2.0) (-0.8,-1.2) 1 2 The numbered white boxes show approximately where the inner loop was called The point in red brackets is where APPSPACK is before the inner loop call The point in green was found by the inner loop 2 (-0.56,1.6) (-0.61,1.25)

26. Groundwater Remediation

27. Groundwater Remediation via Optimization  Optimization techniques can aid the design process to result in lower clean up costs.  Use Hydraulic Capture (HC) models to alter the groundwater flow direction and control plume migration 1. Transport Based Concentration Control (TBCC) Computationally expensive Computationally expensive Well defined plume boundary Well defined plume boundary → MFO high fidelity model 2. Flow Based Hydraulic Control (FBHC) Orders of magnitude faster Orders of magnitude faster Constraints require calibration Constraints require calibration → MFO low fidelity model

28. Optimization  Objective Function J(u) = installation costs + operation costs J(u) = installation costs + operation costs Evaluation requires results of a simulation Evaluation requires results of a simulation  Design Variables Number of wells Number of wells Well pumping rates Well pumping rates Well locations Well locations  Constraints Well capacity Well capacity Net pumping rate Net pumping rate Don’t flood or dry out land Don’t flood or dry out land No useless wells No useless wells  Implementation Derivatives are unavailable  Simulators MODFLOW: used for flow equation (USGS) MT3D: used for transport equation (EPA)

29. MFO Numerical Test  Test the MFO method on the HC problem included in the community problems set. (Mayer, Kelley, Miller)  The FBHC formulation has been shown to be sufficient for this simple domain. (Fowler, Kelley, Kees, Miller)  Other approaches are needed for heterogeneous more realistic domains. (Ahlfeld, Page, Pinder)

30. MFO Results MethodCost % Decrease # Fn Evals FBHC$24,17669.2% 117 mf2k 0 mt3d TBCC$20,36274.1% 188 mf2k 160 mt3d MFO$22,42871.5% 152 mf2k 152 mf2k 86 mt3d 86 mt3d Initial cost: $78,586 MODFLOW (mf2k): ~2 seconds mt3d: ~50 seconds

31. MFO Results

32. Acknowledgements  MFO development team (Sandia) Joe Castro (PI), Electrical & Microsystem Modeling, NM Joe Castro (PI), Electrical & Microsystem Modeling, NM Tony Giunta, Validation & Uncertainty Quantification Processes, NM Tony Giunta, Validation & Uncertainty Quantification Processes, NM Patty Hough, CSMR, CA Patty Hough, CSMR, CA  Groundwater Application Katie Fowler, Clarkson University  Questions?? Genetha Gray gagray@sandia.gov  Software APPSPACK:APPSPACK: software.sandia.gov/appspack/version4.0/index.html DAKOTA:DAKOTA: http://endo.sandia.gov/DAKOTA