Science in the 20th century Materials and devices Relativity Quantum mechanics Chemical bond Molecular basis of life Systems Robustness (Bode, Zames,…) Computational complexity (Turing, Godel, …) Information (Shannon, Kolmogorov) Chaos and dynamical systems (Poincare, Lorenz,…) Optimal control (Pontryagin, Bellman,…)

Current dominant challenges Materials and devices Unified field theory Dynamics of chemical reactions Dynamics of biological macromolecules Systems Robustness of complex interconnected dynamical systems and networks “Unified field theory” of control, communications, computing

Current dominant challenges Robustness of complex interconnected dynamical systems and networks Role of control theory Robustness Interconnection Rigor We need an expanded view of all of these.

Robust Humans have exceptionally robust systems for vision and speech.

Yet fragile …but we’re not so good at surviving, say, large meteor impacts.

Yet fragile …but we’re not so good at surviving, say, large meteor impacts.

Robustness and uncertainty Error, sensitivity Types of uncertainty Sensitive Robust

Robustness and uncertainty Error, sensitivity Types of uncertainty speech/ vision Meteor impact Humans Archaebacteria Sensitive Robust

yet fragile Error, sensitivity Types of uncertainty speech/ vision Meteor impact Humans Archaebacteria Sensitive Robust Robustness and uncertainty

Robust yet fragile Error, sensitivity Types of uncertainty “Complex” systems Sensitive Robust

“Robust, yet fragile” Robust to uncertainties –that are common, –the system was designed for, or –has evolved to handle, …yet fragile otherwise This is the most important feature of complex systems (the essence of HOT).

Example: Auto airbags Reduces risk in high- speed collisions Increases risk otherwise Increases risk to small occupants Mitigated by new designs with greater complexity Could just get a heavier vehicle Reduces risk without the increase! But shifts it elsewhere: occupants of other vehicles, pollution

Biology (and engineering) Grow, persist, reproduce, and function despite large uncertainties in environments and components. Yet tiny perturbations can be fatal –a single specie or gene –minute quantities of toxins Complex, highly evolved organisms and ecosystems have high throughput, But are the most vulnerable in large extinctions. Complex engineering systems have similar characteristics

Types of uncertainty Automobile air bags Error, sensitivity high speed head-on collisions children low-speed

Information/ Computation Robustness/ Uncertainty Is robustness a “conserved quantity”? Materials Energy Entropy constrained

Uncertainty and Robustness Complexity Interconnection/ Feedback Dynamics Hierarchical/ Multiscale Heterogeneous Nonlinearity

Uncertainty and Robustness Complexity Interconnection/ Feedback Dynamics Hierarchical/ Multiscale Heterogeneous Nonlinearity

uncertain sequence - error delaypredictor Prediction: the most basic scientific question.

e(k) = x(k) - u(k-1) x(k) : uncertain sequence - e(k) delaypredictor u(k-1)u(k) u(k) : prediction of x(k+1) e(k) : error x(k) k u(k) e(k) k

- ex PlantControl c feedforward - e x PlantControl c feedback Prediction is a special case of feedforward. For known stable plant, these are the same:

x(k) k e(k) k u(k) For simplicity, assume x, u, and e are finite sequences. Then the discrete Fourier transform X, U, and E are polynomials in the transform variable z. If we set z = e i ,  then X(  ) measures the frequency content of x at frequency .

e(k) = x(k) - u(k-1) x e - delay C u(k-1)u(k) x(k) e(k) u(k) How do we measure performance of our predictor C in terms of x, e, X, and E? Typically want ratios of norms: or whereand are some suitable time or frequency domain (semi-) norms, usually weighted. to be small.

Good performance (prediction) means or Equivalently, or For example, Plancheral Theorem:

Interesting alternative: Not a norm, but a very useful measure of signal size, as we’ll see. (The b in ||  || b is in honor of Bode.) Or to make it closer to existing norms:

A useful measure of performance is in terms of the sensitivity function S(z) defined by Bode as If we set z = e i ,   then |S(  )| measures how well C does at each frequency. (If C is linear then S is independent of x, but in general S depends on x.) It is convenient to study log |S(  )| and then u   0 ( u(k)=0  k)  S  1, and log|S|  0. log |S(  )| > 0  C amplifies x at frequency . log |S(  )| < 0  C attenuates x at frequency .

x(k) k u(k-1) Assume u is a causal function of x. Note: as long as we assume that for any possible sequence {x(k)} it is equally likely that {-x(k)} will occur, then guessing ahead can never help. Then the first nonzero element of u is delayed at least one step behind the first nonzero element of x. This implies that (taking z  ) 0 This will be used later.

e(k) = x(k) - u(k-1) - e(k) delay C u(k-1) x(k) u(k) For any C, an unconstrained “worst-case” x(k) is x(k) = -u(k-1), which gives e(k) = x(k) - u(k-1) = - 2*u(k-1) = 2*x(k) Thus, if nothing is known about x(k), the “safest” choice is u   0. Any other choice of u does worse in any norm. If x is white noise, then u   0 is also the best choice for optimizing average behavior in almost any norm.

Summary so far: Some assumptions must be valid about x in order that it be at all predictable. Intuitively, there appear to be fundamental limitations on how well x can be predicted. Can we give a precise mathematical description of these limitations that depends only on causality and require no further assumptions? e(k) = x(k) - u(k-1) - e(k) delay C u(k-1) x(k) u(k)

If x is chosen so that X(z) has no zeros in |z| > 1 (this is an open set), then Proof: Follows directly from Jensen’s formula, a standard result in complex analysis (advanced undergraduate level). Recall that S(z) = E(z)/X(z) and S(  ) = 1. Denote by {  k } and {  k } the complex zeros for |z| > 1 of E(z) and X(z), respectively. Then

If the predictor is linear and time-invariant, then Under some circumstances, a time-varying predictor can exploit signal “precursors” that create known {  k } Recall that S(z) = E(z)/X(z) and S(  ) = 1. Denote by {  k } and {  k } the complex zeros for |z| > 1 of E(z) and X(z), respectively.

log|S | > 0 amplified log|S | < 0 attenuated  log|S |  he amplification must at least balance the attenuation.

 Robust …yet fragile

Originally due to Bode (1945). Well known in control theory as a property of linear systems. But it’s a property of causality, not linearity. Many generalizations in the control literature, particularly in the last decade or so. Because it only depends on causality, it is in some sense the most fundamental known conservation principle. This “conservation of robustness” and related concepts are as important to complex systems as more familiar notions of matter, energy, entropy, and information.

Recall: is equivalent to

Uncertainty and Robustness Complexity Interconnection/ Feedback Dynamics Hierarchical/ Multiscale Heterogeneous Nonlinearity

- ex PlantControl c feedforward - e x PlantControl c feedback What about feedback?

Positive ( F > 0 ) ln(S) > 0 disturbance amplified Negative ( F < 0) ln(S) < 0 disturbance attenuated Simple case of feedback. F + e d c e = error d = disturbance c = control e = d + c = d + F (e) (1-F )e = d If e, d, c, and F are just numbers: S = sensitivity function measures the disturbance rejection It’s convenient to study ln(S).

ln(S) F F < 0 ln(S) < 0 attenuation F > 0 ln(S) > 0 amplification

ln(S) F F   ln(S)   extreme robustness extreme robustness F  1 ln(S)   extreme sensitivity extreme sensitivity

Uncertainty and Robustness Complexity Interconnection/ Feedback Dynamics Hierarchical/ Multiscale Heterogeneous Nonlinearity

If these model physical processes, then d and e are signals and F is an operator. We can still define S(  = | E(  /D(  | where E and D are the Fourier transforms of e and d. ( If F is linear, then S is independent of D.) Under assumptions that are consistent with F and d modeling physical systems (in particular, causality), it is possible to prove that: F + e d c

log|S | > 0 amplified log|S | < 0 attenuated  log|S |  he amplification must at least balance the attenuation. Positive and negative feedback are balanced.

 log|S | ln|S| F Positive feedback Negative feedback

 log|S | ln|S| F Positive feedback Negative feedback Robust …yet fragile

Formula 1: The ultimate high technology sport Feedback is very powerful, but there are limitations. It gives us remarkable robustness, as well as recursion and looping. But can lead to instability, chaos, and undecidability.

Cruise control Electronic ignition Temperature control Electronic fuel injection Anti-lock brakes Electronic transmission Electric power steering (PAS) Air bags Active suspension EGR control In development: drive-by-wire steering/traction control collision avoidance

Formula 1 allows: Electronic fuel injection Computers Sensors Telemetry/Communications Power steering sensors computers actuators driver telemetry No active control allowed.

Control Theory Statistical Physics Dynamical Systems Information Theory Computational Complexity Theory of Complex systems?

uncertain sequence - error delaypredictor This is a natural departure point for introduction of chaos and undecidability. F + e d c Kolmogorov complexity Undecidability Chaos Probability, entropy Information Bifurcations, phase transitions