Computational Methods for Design Lecture 4 – Introduction to Sensitivities John A. Burns C enter for O ptimal D esign A nd C ontrol I nterdisciplinary C enter for A pplied M athematics Virginia Polytechnic Institute and State University Blacksburg, Virginia A Short Course in Applied Mathematics 2 February 2004 – 7 February 2004 N∞M∞T Series Two Course Canisius College, Buffalo, NY

A Falling Object “Newton’s Second Law”. y(t) { { AIR RESISTANCE

System of Differential Equations

Parameters IN REAL PROBLEMS THERE ARE PARAMETERS SOLUTIONS DEPEND ON THESE PARAMETERS WE WILL BE INTERESTED IN COMPUTING SENSITIVITIES WITH RESPECT TO THESE PARAMETERS

Examples: n=m=1 CONTINUOUS EVERYWHERE UNIQUE SOLUTION CONTINUOUS WHEN UNIQUE SOLUTION

Logistics Equation

Computing Sensitivities HOW DO WE COMPUTE THE SENSITIVITIES …

SEM Example 1 DIFFERENTIATE

SEM Example 1

SEM Method SOLVE THE SYSTEM (DE) – (SE) (DE) (SE) 1.WHY DO IT THIS WAY ? 2.WE DERIVED (SE) BY USING THE KNOWN SOLUTION … HOW DO WE FIND (SE) IN GENERAL? 3.HOW GENERAL IS THIS PROCESS?

Derivation of SEN Eq DIFFERENTIATE THE EQUATION WITH RESPECT TO q INTERCHANGE THE ORDER OF DIFFERENTIATION

Derivation of SEN Eq (DE) (SE)

SEM Method (DE) (SE)

Explicit Euler for SEQs t0t0 (x 0,s 0 ) R2R2

Explicit Euler for SEQs

SOLVE BOTH DE AND SE TOGETHER HOW DOES IT WORK?

MATLAB Code for SEM Set q Set x 0 and s 0 Set h Time interval Set ICs Explicit Euler

DE Solution x(t)

SE Solution s(t)

Special Structure of SE’s (DE) (SE) (DE) (SE) FIRST: SOLVE (DE) SECOND: SOLVE (SE)

Logistics Equation

SEQ for the Logistics Equation DIFFERENTIATE THE EQUATION WITH RESPECT TO q 1

SEQ for the Logistics Equation

INTERCHANGE THE ORDER OF DIFFERENTIATION

SEQ for the Logistics Equation

NEED SENSITIVITY WITH RESPECT TO q 2

SEQ for the Logistics Equation 2 DIFFERENTIATE THE EQUATION WITH RESPECT TO q 2

SEQ for the Logistics Equation 2

INTERCHANGE THE ORDER OF DIFFERENTIATION

SEQ for the Logistics Equation 2

SEQ’s for the Logistics Equation FROM THE FIRST PARTIAL THE LOGISTICS EQUATION

SEQ’s for the Logistics Equation FIRST: SOLVE (DE) SECOND: SOLVE (SEs)

Model Problem #1 q q q SENSITIVITY The sensitivity equation for s(x, q ) =  q w(x, q ) in the “ physical ” domain  ( q ) = (0, q ) is given by Can be made “ rigorous ” by the method of mappings. MORE ABOUT THIS NEAR THE END q q q  q,q, q q q

Typical Cost Function WHERE w( x, q ) USUALLY SATISFIES A DIFFERENTIAL EQUATION AND q IS A PARAMETER (OR VECTOR OF PARAMETERS) q q q q q q q q q q q q q q THE CHAIN RULE PRODUCES OR (Reality) USING NUMERICAL SOLUTIONS hh hh CONTINUOUS SENSITIVITY DISCRETE SENSITIVITY

Computing Gradients (I) BY FINITE DIFFERENCES q q0q0 q0q0 q0q0 q h h h TYPICAL APPROACHES TO COMPUTE q q =q0q =q0 h (II) BY DISCRETE SENSITIVITIES q0q0 q0q0 q0q0 q0q0 q0q0 h h h h

Computing Gradients FINITE DIFFERENCES REQUIRES 2 NON-LINEAR SOLVES IF SHAPE IS A DESIGN VARIABLE, FD REQUIRES 2 MESH GENERATIONS DISCRETE SENSITIVITIES REQUIRES THE EXISTENCE OF THE DISCRETE SENSITIVITY IF SHAPE IS A DESIGN VARIABLE, THE DISCRETE SENSITIVITY LEADS TO MESH DERIVATIVES COMPUTATIONS WHAT IS THE “ CONTINUOUS / HYBRID ” SENSITIVITY EQUATION METHOD? --- SEM q0q0 q0q0 q0q0 h h h, k APPROXIMATE

A Sensitivity Equation Method FOR q > 1 AND h=q / (N+1) CONSIDER (FORMAL ) h h h h h NUMERICAL APPROXIMATION x=0 x=1 x=q x w(x) w h (x) = Finite Element Approximation q q,q,  DISCRETE STATE EQUATION

A Sensitivity Equation Method h h h h h h h q q,q,  q q  q,q, q h  IMPORTANT OBSERVATIONS l The sensitivity equations are linear The sensitivity equation “ solver ” can be constructed independently of the forward solver -- SENSE™ When done correctly “ mesh gradients ” are not required

A Sensitivity Equation Method FOR q > 1 AND k = q / (M+1) CONSIDER (FORMAL) 2 nd NUMERICAL APPROXIMATION x=0 x=1 x=q x s(x)=  q w(x,q) s h,k (x) = Finite Element Approximation of  h h,k q q  q,q, q q q h h h

Convergence Issues q q q h h h q k h,k THEOREM. The finite element scheme is asymptotically consistent. qq h h h k k a trust region method should (might?) converge. qq h h k When the erroris small, then IDEA: J. T. Borggaard and J. A. Burns, “ A PDE Sensitivity Equation Method for Optimal Aerodynamic Design ”, Journal of Computational Physics, Vol.136 (1997), R. G. Carter, “ On the Global Convergence of Trust-Region Algorithms Using Inexact Gradient Information ”, SIAM J. Num. Anal., Vol 28 (1991), J. T. Borggaard, “ The Sensitivity Equation Method for Optimal Design ”, Ph.D. Thesis, Virginia Tech, Blacksburg, VA, 1995.

Convergence Issues N=16, M=32

Convergence Issues qq h hh THE CASE k = h is often used, but may not be “ good enough ” NOT CONVERGENT

Timing Issues THE CASE k = 2h offers flexibility and qq h h 2h2h convergence. But, what about timings? Approximately 96.6% of cpu time spent in function evaluations Approximately 02.4% of cpu time spent in gradient evaluations

Mathematics Impacts “ Practically ” UNDERSTANDING THE PROPER MATHEMATICAL FRAMEWORK CAN BE EXPLOITED TO PRODUCE BETTER SCIENTIFIC COMPUTING TOOLS  A REAL JET ENGINE WITH 20 DESIGN VARIABLES l PREVIOUS ENGINEERING DESIGN METHODOLOGY REQUIRED 8400 CPU HRS ~ 1 YEAR l USING A HYBRID SEM DEVELOPED AT VA TECH AS IMPLEMENTED BY AEROSOFT IN SENSE™ REDUCED THE DESIGN CYCLE TIME FROM CPU HRS ~ 1 YEAR TO 480 CPU HRS ~ 3 WEEKS NEW MATHEMATICS WAS THE ENABLING TECHNOLOGY

Special Structure of SE’s (DE) (SE) (DE) (SE) FIRST: SOLVE (DE) SECOND: SOLVE (SE)

END