1 Introduction to Model Order Reduction Thanks to Jacob White, Peter Feldmann II.1 – Reducing Linear Time Invariant Systems Luca Daniel

2 Model Order Reduction Linear Time Invariant Systems II.1.a via Modal Analysis II.1.b via Rational Function Fitting (point matching) II.1.c. via Quasi Convex Optimization II.1.d via Pade’ approximation and AWE

3 Introduction to Model Order Reduction Thanks to Jacob White, Peter Feldmann II.1.a – Reduction using Modal Analyis Luca Daniel

4 State-Space Description Dynamic Linear case Original Dynamical System - Single Input/Output Reduced Dynamical System q << N, but input/output behavior preserved

5 Defining Accuracy Time-domain response should be “close”Time-domain response should be “close” –For which possible inputs? Frequency response should matchFrequency response should match –At what frequencies?

6 Matching Frequency Response Ensure accuracy for only some inputs?Ensure accuracy for only some inputs? Example:Example: –low frequency inputs, –or some band, –or some points in the frequency response Original matching some part of the frequency response

7 Reminder about Eigenanalysis

8 Reminder about Eigenanalysis Cont. Decoupled Equations Output Equation

9 Reminder about Eigenanalysis Cont. Solving Decoupled Equations Output Equation Assuming Zero Initial Conditions

10 Reduced models via mode truncation Dynamic Linear Case Output Equation

11 Reduced models via mode Truncation Dynamic Linear Case Why? Certain modes are not affected by the input Certain modes do not affect the output Keep least negative eigenvalues (slowest modes) –Look at response to a constant input

12 Reduced models via mode truncation Dynamic Linear Case Heat Conducting bar Results N=100 Exact q=1 q=3 q=10 Keep qth slowest modes

13 Another way to look at Reduction by Modal Analysis Transfer Function Apply Eigendecomposition elimitate each mode for which this term is small

14 Model Order Reduction via Eigenmode Analysis Pole-Residue Form Pole-Zero Form (SISO) Ideas for reducing order:Ideas for reducing order: –Drop terms with small residues –Drop terms with large negative (“fast” modes) –Remove pole/zero near-cancellations –Cluster poles that are “together”

15 Eigenmode Analysis Based Reduction Summary AdvantagesAdvantages –Conceptually familiar –Simple physical interpretation : retains dominant system modes/poles DrawbacksDrawbacks –Relatively expensive : have to find the eigenvalues first –Relatively inefficient. For a given model size, many other approaches can provide better accuracy for the same computational cost e.g. Hankel Model Order Reductione.g. Hankel Model Order Reduction e.g. Truncated Balance Realizatione.g. Truncated Balance Realization O(n 3 )

16 Model Order Reduction Linear Time Invariant Systems II.1.a via Modal Analysis II.1.b via Rational Function Fitting (point matching) II.1.c. via Quasi Convex Optimization II.1.d via Pade’ approximation and AWE

17 Introduction to Model Order Reduction Thanks to Jacob White II.1.b – Reduction using Fitting Luca Daniel

18 A canonical form for model order reduction Assuming A is non- singular we can cast the dynamical linear system into one canonical form for model order reduction Note: not necessarily always the best, but the simplest for educational purposes

19 Original System Transfer Function: Model Reduction = Find a low order (q << N) rational function matching rational function matching Model Order Reduction via Rational Transfer Function Fitting rational function reduced order rational function

20 Reduced Model Dynamical System Reduced Model Transfer Function coefficients coefficients Rational Transfer Function Fitting: Degrees of Freedom

21 Reduced Model Transfer Function Apply any invertible change of variables to the state Many Dynamical Systems have the same transfer function!! Rational Transfer Function Fitting: Degrees of Freedom (cont.) I I

22 Rational Transfer Function Fitting: via Point Matching For i = 1 to 2q cross multiplying generates a linear systemcross multiplying generates a linear system Can match 2q pointsCan match 2q points

23 Columns contain progressively higher powers of the test frequencies: problem is numerically ill-conditionedColumns contain progressively higher powers of the test frequencies: problem is numerically ill-conditioned also... missing data can cause severe accuracy problemsalso... missing data can cause severe accuracy problems Rational Transfer Function Fitting: Point Matching matrix can be ill-conditioned

SMA 2005 MIT 24 Hard to Solve Systems Fitting Example Polynomial Interpolation Table of Data t 0 f (t 0 ) t 1 f (t 1 ) t N f (t N ) f t t0t0 t1t1 t2t2 tNtN f (t 0 ) Problem fit data with an N th order polynomial

SMA 2005 MIT 25 Hard to Solve Systems Example Problem Matrix Form

SMA 2005 MIT 26 Hard to Solve Systems Fitting Example Coefficient Value Coefficient number Fitting f(t) = t f t

SMA 2005 MIT 27 Hard to Solve Systems Perturbation Analysis Geometric Approach is clearer When vectors are nearly aligned, difficult to determine how much of versus how much of Case 1 Columns orthogonal Columns nearly aligned

SMA 2005 MIT 28 Hard to Solve Systems Geometric Analysis Polynomial Interpolation ~314 ~10 6 ~10 13 ~10 20 log(cond(M)) n The power series polynomials are nearly linearly dependent t t 2 t

29 Course Outline Numerical Simulation Quick intro to PDE Solvers Quick intro to ODE Solvers Model Order reduction Linear systems Common engineering practice Optimal techniques in terms of model accuracy Efficient techniques in terms of time and memory Non-Linear Systems Parameterized Model Order Reduction Linear Systems Non-Linear Systems Yesterday Today Friday Thursday Tomorrow

30 Introduction to Model Order Reduction Luca Daniel Massachusetts Institute of Technology

31 Course Outline Numerical Simulation Quick intro to PDE Solvers Quick intro to ODE Solvers Model Order reduction Linear systems Common engineering practice Optimal techniques in terms of model accuracy Efficient techniques in terms of time and memory Non-Linear Systems Parameterized Model Order Reduction Linear Systems Non-Linear Systems Monday Yesterday Friday Tomorrow Today

32 Model Order Reduction Linear Time Invariant Systems II.1.a via Modal Analysis II.1.b via Ratianal Function Fitting (point matching) II.1.c. via Quasi Convex Optimization II.1.d via Pade’ approximation and AWE

33 Introduction to Model Order Reduction Thanks to Kin C. Sou, Alexander Megretski II.1.c – Reduction using Optimization Luca Daniel

34 Overview Optimization based reduction Quasi-convex optimization MOR setup Solving the MOR setup Application examples Conclusions

35 Recall Rational Transfer Function Fitting via Point Matching For i = 1 to 2q cross multiplying generates a linear systemcross multiplying generates a linear system Can match 2q pointsCan match 2q points

36 Optimization based rational fit Model Order Reduction Setup From field solver Or measurements Small stable and passive reduced order model Least Square method Cast as nonlinear least squares (solved by Gauss- Newton) Quasi-convex method Cast as quasi-convex program (solved by convex optimization algorithm) Do not consider stability or passivity while finding poles (need post- processing) Explicitly take care of stability and passivity while finding poles

37 Change of variables To make our program tractable, we introduce a change of frequency variables (bilinear transform) Laplace frequency variable z frequency variable [s] [z]

38 Desirable MOR setup to solve Feasible set is not convex if m  3 For example, but Problem has not been proved to be NP complete either Modified optimal H-inf norm MOR setup Stability: q(z) Schur polynomial (roots inside unit circle) Passivity, and possibly other constraints

39 Overview Optimization based reduction Quasi-convex optimization MOR setup Solving the MOR setup Application examples Conclusions

40 Relaxation Original problem is difficult Made easier if some constraints are dropped (relaxed) Solve the relaxed problem Construct original solution from relaxation For example, LP relaxation (polynomial time) of IP problems (exponential time). General idea -c feasible set … optimal solution -c feasible set optimal relaxed solution nearest rounding

41 Relaxation of the H-inf norm MOR setup Benefit: Relaxation equivalent to a quasi-convex program. Drawback: May obtain suboptimal solutions Anti-stable term Stability: q(z) Schur polynomial (roots inside unit circle) Passivity, and possibly other constraints

42 How bad is this relaxation? Let such that deg(q) = m, q(z) is Schur polynomial Then m+1 th Hankel singular value THEOREM:

43 Change of variables where a(z) b(z) and c(z) are trigonometric polynomials: when Prop: Stability 

44 Passivity For SISO systems, passivity means 1.H(z) is analytic for |z|>=1 2.H(z)*=H(z*) 3.Re(H(z))>0 for |z|=1 for impedance, Conclusion: Stability and passivity = positivity of trigonometric polynomials for all frequencies!

45 Equivalent quasi-convex setup This is a quasi-convex program, because defines an intersection of halfspaces and  sub-level set is is again intersection of halfspaces parameterized by  and  convex set 00 11 =0=0 =1=1 =2=2 =3=3 quasi-convex function convex set

46 Additional constraints Can model additional constraints such as Bounded real passivity (for scatter parameters) Explicit minimization of quality factor error (for inductors) Weighting of frequency responses Point-wise transfer function (and/or derivatives) matching

47 Overview Optimization based reduction Quasi-convex optimization MOR setup Algorithm Summary Application examples Conclusions

48 Summary of QCO algorithm Step 2: Compute coefficients of q(z) using the relation and q(z) being a Schur polynomial Step 3: Compute coefficients of p(z) by solving Solved for example by the ellipsoid algorithm Step 1: Compute optimal solution a(z),b(z),c(z) of the relaxation subject to stability, passivity…,stability, passivity… Solved for example by the ellipsoid algorithm

49 Stability? Solving quasi-convex programs (a,b,c,  ) current iterate localization set (e.g. ellipsoid) Passivity? … Generate cut N N N N Y Y Y Decrease  All Yes Termination? N target set localization set center cut min volume covering ellipsoid new center new cut and so on Update localization set Objective oracle, stability oracle, passivity oracle…

50 Overview Optimization based reduction Quasi-convex optimization MOR setup Algorithm summary Application examples Conclusions

51 Example 1: RLC line (MNA) “PRIMA” (Moment Matching) Model Order Reduction Quasi Convex Optimization Model Order Reduction RLC line full model 20 th order [Vasilyev 2004] Open circuit terminal 10 th order reduced model by existing PRIMA and our QCO 4 4 2

52 Example 2: RF inductor with substrate (from field solver) RF inductor with substrate effect captured by layered Green’s function [Hu Dac 05] System matrices are frequency dependent Full model has infinite order Reduced model has order 6

53 Example 3: RF inductor model (from measurement) Fabricated 7 turn spiral inductor Blue: measurement Red: 10 th order reduced model (positive real part constraint imposed)

54 Example 4: Model of graphic card package (from measurement) Industry example of a multi-port device (390 frequency samples) 12 th order SISO reduced models are constructed Bounded realness constraint is imposed Frequency weight is employed S11 S13 Solid: ROM Dot: measurement Solid: ROM Dot: measurement

55 Example 5: Large IC power distribution grid (from field solver) Power distribution grid (dimension size = 7mm, wire width = 2 µm) Blue: full model (order 2046) Green: PRIMA 40 th order reduced model Red: QCO 40 th order reduced model (positive real) 3 curves on top of each other

56 Conclusion QCO competes reasonably well in terms of accuracy with moment matching (e.g. PRIMA) for reducing large systems But in addition: QCO can reduce models with frequency dependent matrices QCO is very flexible in imposing constraints such as stability and passivity QCO can be extended to parameterized MOR problems (see IV.2)

57 Model Order Reduction Linear Time Invariant Systems II.1.a via Modal Analysis II.1.b via Ratianal Function Fitting (point matching) II.1.c. via Quasi Convex Optimization II.1.d via Pade’ approximation and AWE

58 Point matching vs. Moment Matching Point matching: can be very inaccurate in between points Moment (derivatives) matching: accurate around expansion point, but inaccurate on wide frequency band

59 The Taylor coef. = frequency domain moments = = derivatives of the transfer function (up to a constant) Frequency Domain "Moments" (or Taylor coefficients) of the transfer function Taylor Series Expansion of the original transfer function around s=0

60 Time domain moments of the impulse response Definition:

61 Connection to the time-domain moments of the circuit response Time-domain moments Compare: Hence the the Taylor coeff. are, up to a constant, the time-domain moments of the circuit response.

62 Rational function fitting via moment matching: Pade Approximation (AWE)

63 Rational function fitting via moment matching: Pade Approximation (AWE) –Step 1: calculate the first 2q moments of H(s) –Step 2: calculate the 2q coeff. of the Pade’ approx, matching the first 2q moments of H(s)

64 Step 1: calculation of moments simulating equivalent circuits (AWE) Historical noteHistorical note –Electrical engineers calculated freq. domain Taylor coef. by calculating time domain moments, –synthesizing and simulating circuit networks. –Specifically the momets can be calculated evaluating the asymptotic behaviors of the circuit waveforms, –Hence the name AWE (Asymptotic Waveform Evaluation)

65 Step 1: Calculation of moments (algebraically) For sparse systemFor sparse system –can use one initial LU decomposition on A –then solve 2q linear triangular systems for the 2q moments For dense systemsFor dense systems –can use iterative methods and matrix implicit matrix-vector products

66 Step 2: Calculation of Pade’ coeff. (AWE) For coeff. a’s solve the following linear system: For coeff. b’s simply calculate:

67 Heat Conducting Bar Demonstration Example State-Space Description Given the right scaling Heat In

N=100 Exact q=1 q=3 q=10 Keep q th slowest eigenmodes Exact q=1 q=3 Matches q moments Keeping Eigenmodes versus matching moments Dynamic Linear Case Heat Flow Results

69 Vectors will line up with dominant eigenspace! Numerical problem for q >20 (cannot get accuracy) matrix powers converge to the eigenvector corresponding to the largest eigenvalue.

70 Pade matrix can be very ill-conditioned matrix powers converge to the eigenvector corresponding to the largest eigenvalue.matrix powers converge to the eigenvector corresponding to the largest eigenvalue. Columns become linearly dependent for large q the problem is numerically very ill-conditioned!

71 Pade matrix can be very ill-conditioned matrix powers converge to the eigenvector corresponding to the largest eigenvalue.matrix powers converge to the eigenvector corresponding to the largest eigenvalue. Columns become linearly dependent for large q the problem is numerically very ill-conditioned!

72 Example: simulation of voltage gain of a filter with Pade via AWE

73 Model Order Reduction Summary. Linear Time Invariant Systems II.1.a via Modal Analysis II.1.b via Rational Function Fitting (point matching) II.1.c. via Quasi Convex Optimization II.1.d via Pade’ approximation and AWE