Quantitative, Local Analysis for Nonlinear Systems AFOSR: FA9550-05-1-0266, April 2005-April 08 Participants UCB: Ufuk Topcu, Weehong Tan, Packard, Tim Wheeler UMN: Gary Balas Honeywell: Pete Seiler More info http://jagger.me.berkeley.edu/~pack/certify http://jagger.me.berkeley.edu/~pack/certificates Copyright 2008, Packard, Topcu, Tan, Wheeler, Seiler and Balas. This work is licensed under the Creative Commons Attribution-ShareAlike License. To view a copy of this license, visit http://creativecommons.org/licenses/by-sa/2.0/ or send a letter to Creative Commons, 559 Nathan Abbott Way, Stanford, California 94305, USA.

Numerical tools to quantify/certify dynamic behavior August 2008 Summary Numerical tools to quantify/certify dynamic behavior Locally, near equilibrium points Analysis considered Region-of-attraction, input/output gain, reachability, hard IQCs Methodology Enforce Lyapunov/Dissipation inequalities locally, on sublevel sets Set containments via S-procedure and SOS constraints Bilinear semidefinite programs “always” feasible Simulation aids nonconvex proof/certificate search Address model uncertainty Parametric Uncertainty Parameter-independent Lyapunov/Storage Fcn Branch-&-Bound Dynamic Uncertainty Local small-gain theorems Polytope of vector fields Model valid only over known set

Uncertain ICs: Estimating Region of Attraction Dynamics, equilibrium point If there exists positive-definite V with then (Lyapunov) for the ODE, the set is invariant, and in the region-of-attraction. Strategy: Grow estimate via optimization over V, enforcing containments with S-procedure/SOS conditions Challenge: set of certifying Lyapunov functions is not convex.

Estimating Region of Attraction Dynamics, equilibrium point p: Analyst-defined function whose (well-understood) sub-level sets are to be in region-of-attraction By choice of positive-definite V, maximize  so that: Known domain of model validity

Checking Set Containments: S-proc & SOS Consider sets defined via inequalities, using given polynomials pA and pB as Question: Does the set containment constraint hold? Certified by If there exists an SOS ( ) polynomial such that then Linearly parametrize class to search (for s) over as Find so SDP in η and variables which parametrize Σ

Region of Attraction: Bilinear SOS Maximize  (positive-definite V ) so that Choose “small” positive definite functions Remark: If l2 is quadratic, then feasibility implies that the linearized dynamics are exponentially stable. BMIs SDP iteration or direct BMI solver Similar expressions for local L2 gain and reachability Products of decision variables

Summary: Estimating Region of Attraction w/ simulation Dynamics, equilibrium point Find positive-definite V, with Then is invariant, and in the region of attraction of , denoted Given a “shape” function p QUESTION: Fix β>0, is Pragmatic solution: run N sims, starting from samples in If any diverge, then unambiguously “no” If all converge, then maybe “yes”, perhaps Lyapunov analysis to prove/certify it How can we use the simulation data to aid in the nonconvex search for a certifying V? nonconvex constraint on V Simulations yield Collection of convergent trajectories starting in divergent trajectories starting in Necessary cond: If V exists to verify, need V≤1 and decreasing on convergent trajectories V≥0 on all trajectories Quad(V) is a Lyapunov function for Linear(f) V≥1 on the divergent trajectories Linearly parametrize Each candidate V certifies some ROA Assess in 2 steps, using positivity & sum-of-squares (SOS) optimization to enforce subsets SOS optimization (s1, s2) to maximize the level-set condition on V SOS optimization (s3) to maximize condition on p, V Furthermore, coordinatewise solvers are initialized with these, and further optimization (adjusting V too) be performed. Convex constraints in Lyapunov function coefficient-space Sample this convex outer-bound for candidate Lyapunov fcns Necessary conditions are convex constraints on

gain of If there exists positive-definite V with bounded, and then Strategy: enforcing containments with S-procedure/SOS Fix R, shrink estimate of γ, or… Fix γ, grow estimate of R via optimization over V. Use simulation to outer-bound V Exploit constraints on w if available

Uncertain Inputs: Reachability Given a differential equation and a positive definite function p, how large can get, knowing If Then Strategy: Shrink estimate (β) via optimization over V, enforcing containments with S-procedure/SOS conditions. Use simulation and exploit constraints on w if available. Conditions on Conclusion on ODE

Reachability: Refining the bound Positive-definite V, with bounded, satisfying Then Refinement: Suppose there exists a function h, with 0<h≤1, with Define

(s,q) dependence on n and 2d 4 6 8 3 10 27 15 75 20 126 35 465 5 50 420 70 1990 7 28 196 84 2646 210 19152 9 45 540 165 10692 495 109890 11 66 1210 286 33033 1001 457743

Quantitative improvement on linearized analysis Consider dynamics with matrix A is Hurwitz, and function f23 consists of 2nd and 3rd degree polynomials, f23(0)=0 Standard analysis: “there is an open ball around origin in region of attraction” Here… For any quadratic, pos-def l2 These are feasible with some quadratic pos-def l1, using

Quantitative improvement to linearized analysis Consider dynamics with f2, g2, h2 quadratic, f3 cubic, with f2(0,0)=f3(0)=h2(0)=0 matrix A is Hurwitz, and Here… In fact, DIE can be strengthened to include positive-definite term These are always feasible, using

Quantitative improvement to linearized analysis Consider dynamics with functions b bilinear, q quadratic matrix A is Hurwitz Then (eg) the SOS-based local DIE for reachability is always feasible using Marginally-stable linearization at x=0. Common structure of certain adaptive systems

Uncertain Systems: Parameter-Independent V For simplicity, take affine parameter uncertainty Solve earlier conditions, but enforcing at the vertex values of f. Then is invariant, and in the Robust ROA of . Advantages: a robust ROA, and V is only a function of x, δ appears only implicitly through the vertices SOS analysis is only in x variables Simulations are incorporated as before (vary initial condition and δ) Limitations Conservative with regard to uncertainty Conclusions apply to time-varying parameters, hence… often conclusions are too weak for time-invariant parameters polytope in Rm Subdivide Δ Solve separately Δ2 Δ1

Much better: B&B in Uncertainty Space Of course, growth is still exponential in parameters… but kth local problem uses Vk(x) Solve conservative problem over subdomain Local problems are decoupled Trivial parallelization δ1 δ2 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 1 subdivisions 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 3 subdivisions 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 d 2 0.1 0.2 0.3 0.4 0.5 0.6 0.7 0.8 0.9 1 2 subdivisions Non-affine uncertainty, for example Treat as 2 parms on 1-d curve. Cover with polytope. Solve. Refine to union of polytopes. Solve on each polytope. Intersect ROAs → Robust ROA

Generalization of covering manifold Given: polynomial p(δ) in many real variables, Domain , typically a polytope Find a polytope that covers the manifold Tradeoff between number of vertices, and Excess “volume” in polytope One approach: Find “tightest” affine upper and lower bounds over H Enforce with S-procedure linear function of c0, c

Generalization of covering manifold Partition H, repeat For multivariable p, Bound, on H (above and below), each component of p with affine functions, c, d, (e.g, using S-procedure). Then, a covering polytope (Amato, Garofalo, Gliemo) is with 2m+k easily computed vertices.

Unmodeled dynamics Δ C P Fixed structure, parametric Operator P C P C Parametrize fixed-structure system representing unmodeled dynamics Compute certified invariant subset of region-of-attraction valid for all parameter values Just more parameters Operator Introduce perturbation inputs/outputs Compute L2→L2 gain across channel, valid over some L2 ball of inputs. Apply a local small-gain theorem P C xc xp xδ P C xc xp Pδ C Δ Local L2→L2 gain for system with parametric uncertainty. Same approach as ROA.

Unmodeled dynamics: operator Δ Δ C P M Local induced gain constraint (<1) on M Starting from x(0)=0, for all Δ causal, globally stable, Facts:

4-state aircraft example Aircraft: Short period longitudinal model, pitch axis, with 1-state linear controller Simple form for shape factor: Eliminate parameter uncertainty Different Lyapunov function structures Quadratic (βcert=8.6) pointwise-max quadratics (βcert=8.6) Quadratic+Quartic (βcert=12.2) Fully quartic (quadratic + cubic + quartic) βcert=15.3 Other approaches have deficiencies Directly use commercial BMI solver (PENBMI) βcert=15.2, but… 6 hours… SDP iteration from “random” starting point Use P from Initialize 30 iterations, βcert=8.6 4000 simulations 3 minutes Form LP/ConvexP 0.5 minutes Get a feasable point 1 minute Assess answer with V SDP Iterate from V 0.5 min/iteration, 10 iters TOTAL 10 minutes discover divergent trajectories proving Divergent initial condition Certified set of convergent initial conditions Disk in 4-d state space, centered at equilibrium point

4-state aircraft example w/uncertainty Aircraft: Short period longitudinal model, pitch axis, with 1-state linear controller Same form for shape factor: Ad-hoc 9-processor B&B Divide worst region into 9 Not-uncertain results Quadratic (βcert=8.6) Fully quartic (βcert=15.3) Divergent IC,

4-state aircraft example w/uncertainty Δ .75 C 1.25 Pδ Results Nominal with δM, δCG 8.6 (15.3) 5.1 (7.5) With Δ (as operator) 4.2 (6.7) 2.4 (4.1) With Δ (1st order, γ, β) 4.9 2.8 (4.6)

Adaptive System: reachability example analysis Model-reference adaptive systems Example: 2-state P, 2-state ref. model, 3 adaptive parameters Insert additional disturbance (d) not handled by theory Bound worst-case effect of external signals (r,d) on tracking error (e) Initial conditions: r Reference model - Adaptive control plant e Quadratic vector field, marginally stable linearization Input/output gain analysis certifies that for all signals r and d satisfying , error e satisfies for all t. There are particular r and d satisfying causing e to achieve at some time t. -2 -1 1 2 E1 E2

Adaptive System: L2 gain example plant Reference model Adaptive control - 5 4 3 g 2 1 2 4 6 R R

Wrapup/Perspective Tools that handle (cubic, in x, vector field) 15 states, 3 parameters, unmodeled dynamics, analyze with ∂(V)=2 7 states, 3 parameters, unmodeled dynamics, analyze with ∂(V)=4 Certified answers, not clear that these are appropriate for design choices Sproc/SOS/DIE more quantitative than linearization Linearized analysis: quadratic storage functions, infinitesimal sublevel sets SOS/S-procedure always works Can this scale up to large, complex systems analysis (e.g., adaptive flight controls) where “certificates” are desired? Proofs of behavior with certificates Extensive simulation Probabilistic guarantees? and linearized analysis