Download presentation
Presentation is loading. Please wait.
Published byRalf McCormick Modified over 6 years ago
1
Fat tails, hard limits, thin layers John Doyle, Caltech
Rethinking fundamentals* Parameter estimation and goodness of fit measures for fat tail distributions Hard limits on robust, efficient networks integrating comms, controls, energy, materials Essentials of layered architectures, naming and address, pub-sub, control, coding, latency, and implications for wireless Implications for control over networks Try to get you to read some papers you might otherwise not *really simple so we can move fast
2
Systems Fundamentals! A rant
A series of “obvious” observations (hopefully)
3
Case studies Networking and clean slate architectures
wireless end systems info or content centric application layer integrate routing, control, scheduling, coding, caching control of cyber-physical Lots from cell biology glycolytic oscillations for hard limits bacterial layering for architecture Neuroscience PC, OS, VLSI, etc (IT components) Earthquakes Medical physiology Smartgrid, cyber-phys Physics: turbulence, stat mech (QM?) Wildfire ecology Lots of aerospace Fundamentals!
4
Fix? Existing design frameworks Sophisticated components
Poor integration Limited theoretical framework Fix? Meta-layers Physiology Organs Prediction Goals Actions errors Cortex Fast, Limited scope Slow, Broad scope Comms Disturbance Plant Remote Sensor Actuator Interface Control
5
Layered architectures
Meta-layers Physiology Organs Prediction Goals Actions errors Cortex Fast, Limited scope Slow, Broad scope Comms Disturbance Plant Remote Sensor Actuator Interface Control
6
Doyle and Csete, Proc Nat Acad Sci USA, online JULY 25 2011
This paper aims to bridge progress in neuroscience involving sophisticated quantitative analysis of behavior, including the use of robust control, with other relevant conceptual and theoretical frameworks from systems engineering, systems biology, and mathematics. Doyle and Csete, Proc Nat Acad Sci USA, online JULY
7
Requirements on systems and architectures
accessible accountable accurate adaptable administrable affordable auditable autonomy available credible process capable compatible composable configurable correctness customizable debugable degradable determinable demonstrable dependable deployable discoverable distributable durable effective efficient evolvable extensible failure transparent fault-tolerant fidelity flexible inspectable installable Integrity interchangeable interoperable learnable maintainable manageable mobile modifiable modular nomadic operable orthogonality portable precision predictable producible provable recoverable relevant reliable repeatable reproducible resilient responsive reusable robust safety scalable seamless self-sustainable serviceable supportable securable simplicity stable standards compliant survivable sustainable tailorable testable timely traceable ubiquitous understandable upgradable usable
8
Simplified, minimal requirements
accessible accountable accurate adaptable administrable affordable auditable autonomy available credible process capable compatible composable configurable correctness customizable debugable degradable determinable demonstrable dependable deployable discoverable distributable durable effective efficient evolvable extensible failure transparent fault-tolerant fidelity flexible inspectable installable Integrity interchangeable interoperable learnable maintainable manageable mobile modifiable modular nomadic operable orthogonality portable precision predictable producible provable recoverable relevant reliable repeatable reproducible resilient responsive reusable robust safety scalable seamless self-sustainable serviceable supportable securable simplicity stable standards compliant survivable sustainable tailorable testable timely traceable ubiquitous understandable upgradable usable
9
Requirements on systems and architectures
accessible accountable accurate adaptable administrable affordable auditable autonomy available credible process capable compatible composable configurable correctness customizable debugable degradable determinable demonstrable dependable deployable discoverable distributable durable effective efficient evolvable extensible failure transparent fault-tolerant fidelity flexible inspectable installable Integrity interchangeable interoperable learnable maintainable manageable mobile modifiable modular nomadic operable orthogonality portable precision predictable producible provable recoverable relevant reliable repeatable reproducible resilient responsive reusable robust safety scalable seamless self-sustainable serviceable supportable securable simplicity stable standards compliant survivable sustainable tailorable testable timely traceable ubiquitous understandable upgradable usable fragile robust wasteful efficient
10
Robust to uncertainty in environment and components
Efficient in use of real physical resources fragile Want robust efficiency robust efficient wasteful
11
complex simple 2.5d space of systems and architectures fragile robust
wasteful efficient
12
Want to understand the space of systems/architectures
Case studies? fragile Strategies? Hard limits on robust efficiency? Architectures? Want robust and efficient systems and architectures robust efficient wasteful
13
Deep, but fragmented, incoherent, incomplete
Control, OR Comms Kalman Pontryagin Shannon Bode Nash Theory? Deep, but fragmented, incoherent, incomplete Von Neumann Carnot Boltzmann Turing Godel Heisenberg Compute Einstein Physics
14
? fragile? wasteful? Control Comms Each theory one dimension
Shannon Bode Each theory one dimension Tradeoffs across dimensions Assume architectures a priori Progress is encouraging, but… fragile? slow? ? wasteful? Carnot Boltzmann Turing Godel Heisenberg Compute Physics Einstein
15
Technology? fragile At best we get one robust efficient wasteful
16
??? fragile Often neither robust efficient wasteful
17
??? ? ? Bad architectures? gap? Bad theory? fragile robust efficient
wasteful
18
Conservation “laws”? fragile Case studies Sharpen hard bounds
Hard limit Case studies wasteful
19
Control Comms Shannon Bode fragile? slow? wasteful?
20
- - Control Comms Familiar pictures e=d-u d e=d-u d Entropy
delay Disturbance - - u Plant u Capacity C Sense channel Sense/ Encode Decode Control Entropy Sensitivity Circa 1950? Bode Shannon Assume “favorable” delays
21
- Shannon e=d-u d Capacity CS Decode Entropy
delay Disturbance - Capacity CS Sense channel Sense/ Encode Decode Entropy Assume “favorable” delays
22
- Bode e=d-u d unstable pole p Sensitivity u Plant Control
Simplified and dropped constants, slight differences between cont and disc time
23
- - Control Comms Circa 1950? So, anything happened since? e=d-u d
delay Disturbance - - u Plant u Capacity C Sense channel Sense/ Encode Decode Control Circa 1950? So, anything happened since?
24
Layered architectures
Diverse applications TCP IP MAC MAC MAC Switch Pt to Pt Pt to Pt Physical
25
Doyle and Csete, Proc Nat Acad Sci USA, online JULY 25 2011
This paper aims to bridge progress in neuroscience involving sophisticated quantitative analysis of behavior, including the use of robust control, with other relevant conceptual and theoretical frameworks from systems engineering, systems biology, and mathematics. Doyle and Csete, Proc Nat Acad Sci USA, online JULY
26
Proceedings of the IEEE, Jan 2007
Chang, Low, Calderbank, and Doyle
27
Too clever? Diverse applications TCP IP Diverse Physical
28
Layered architectures
Deconstrained (Applications) TCP Constrained Networks IP Deconstrained (Hardware) “constraints that deconstrain” (Gerhart and Kirschner)
29
Original design challenge?
Deconstrained (Applications) Networked OS Trusted end systems Unreliable hardware TCP/ IP Constrained Facilitated wild evolution Created whole new ecosystem complete opposite Deconstrained (Hardware)
30
Layered architectures
Essentials Deconstrained (Applications) Few global variables Don’t cross layers Control, share, virtualize, and manage resources Constrained OS Processing Memory I/O Deconstrained (Hardware)
31
Layered architectures Bacterial biosphere
Deconstrained (diverse) Environments Architecture = Constraints that Deconstrain Catabolism AA Ribosome RNA RNAp transl. Proteins xRNA transc. Precursors Nucl. DNA DNAp Repl. Gene ATP Enzymes Building Blocks Shared protocols Deconstrained (diverse) Genomes
32
almost Inside every cell ATP Macro-layers Precursors Catabolism
AA Enzymes Nucl. Macro-layers Building Blocks AA transl. Proteins ATP Ribosome RNA transc. xRNA Crosslayer autocatalysis RNAp DNA Repl. Gene DNAp
33
What makes the bacterial biosphere so adaptable?
Environment Deconstrained phenotype Action Core conserved constraints facilitate tradeoffs Catabolism AA Ribosome RNA RNAp transl. Proteins xRNA transc. Precursors Nucl. DNA DNAp Repl. Gene ATP Enzymes Building Blocks Layered architecture Active control of the genome (facilitated variation) Deconstrained genome
34
Layered architectures
Deconstrained (Applications) Few global variables Don’t cross layers Direct access to physical memory? Control, share, virtualize, and manage resources Constrained OS Processing Memory I/O Deconstrained (Hardware)
35
Few global variables Don’t cross layers Bacterial biosphere
Deconstrained (diverse) Environments Few global variables Architecture = Constraints that Deconstrain Catabolism AA Ribosome RNA RNAp transl. Proteins xRNA transc. Precursors Nucl. DNA DNAp Repl. Gene ATP Enzymes Building Blocks Shared protocols Don’t cross layers Deconstrained (diverse) Genomes
36
Problems with leaky layering
Modularity benefits are lost Global variables? Poor portability of applications Insecurity of physical address space Fragile to application crashes No scalability of virtual/real addressing Limits optimization/control by duality?
37
Fragilities of layering/virtualization
Hijacking, parasitism, predation Universals are vulnerable Universals are valuable Breakdowns/failures/unintended/… not transparent Hyper-evolvable but with frozen core
38
Original design challenge?
Deconstrained (Applications) Networked OS Trusted end systems Unreliable hardware TCP/ IP Constrained Facilitated wild evolution Created whole new ecosystem complete opposite Deconstrained (Hardware)
39
Layered architectures
Deconstrained (Applications) Few global variables? Don’t cross layers? TCP/ IP Control, share, virtualize, and manage resources Constrained I/O Comms Latency? Storage? Processing? Deconstrained (Hardware)
40
IP addresses interfaces (not nodes)
IPC App App DNS Global and direct access to physical address! Robust? Secure Scalable Verifiable Evolvable Maintainable Designable … IP addresses interfaces (not nodes) Dev2 Dev2 Dev CPU/Mem Dev2 CPU/Mem CPU/Mem
41
Naming and addressing need to be resolved within layer
translated between layers not exposed outside of layer Physical IP TCP Application Related “issues” VPNs NATS Firewalls Multihoming Mobility Routing table size Overlays …
42
? Next layered architectures Deconstrained (Applications)
Few global variables Don’t cross layers ? Control, share, virtualize, and manage resources Constrained Comms Memory, storage Latency Processing Cyber-physical Deconstrained (Hardware)
43
Persistent errors and confusion (“network science”)
Architecture is least graph topology. Every layer has different diverse graphs. Application Architecture facilitates arbitrary graphs. TCP IP Physical
44
- What happened to this picture? Source d Decode Source coding Code
Hard limits Achievability Decomposition/Layering Channel coding Decode Channel Code
45
Decoupled Hides details Virtualizes channel Decode Code Channel coding
Under certain assumptions Decoupled Hides details Virtualizes channel Decode Code Channel coding Physical layer Rcv Channel Xmit
46
- Source d Decomp Source coding Compress Decoupled Hides details
Virtualizes source Under certain assumptions
47
gap? Hard tradeoffs R Rate distortion (backwards) error
Architecture= separation + coding Hard tradeoffs data rate R
48
gap? delay? Hard tradeoffs R Rate distortion (backwards) error
Architecture= separation + coding Hard tradeoffs data rate R
49
Internet= Distributed OS
Source - Decode Channel Code d Compress Decomp Link layer Rcv Xmit Internet= Distributed OS Layered architecture Control theory Data compression
50
Control/optimization theory needed for
Source - d Compress Decomp Application layer Control/optimization theory needed for Routing Congestion control Scheduling Caching? Distributed control of cyber-physical Still incomplete, needs more integration, with OS, languages Info theory, particularly for wireless Decode Channel Code Physical layer Rcv Xmit
51
- - What next? Control Comms e=d-u d e=d-u d Entropy Sensitivity
delay Disturbance - - u Plant u Capacity C Sense channel Sense/ Encode Decode Control Entropy Sensitivity Circa 1950? What next?
52
- costs Bode e=d-u d unstable pole p u Plant Control causality
stabilize benefits costs unstable pole p
53
- costs Bode e=d-u d unstable pole p u Plant Control causality
benefits stabilize costs unstable pole p
54
- costs Bode e=d-u d unstable pole p u Plant Control causality
benefits stabilize costs unstable pole p
55
- costs e=d-u d Control Channel Plant Control remote control benefits
Martins and Dahleh, IEEE TAC, 2008 - e=d-u d Control Channel Plant Control remote control benefits stabilize max feedback costs
56
- Shannon e=d-u d Capacity CS Entropy Assume “favorable” delays delay
Disturbance - Capacity CS Sense channel Sense/ Encode Decode Entropy Assume “favorable” delays
57
- CS -CS Shannon e=d-u d delay Disturbance Sense/ Decode Encode
benefits -CS
58
- CS -CS costs delay Disturbance e=d-u d Plant Control Sense/ Encode
remote control benefits stabilize -CS Martins, Dahleh, Doyle IEEE TAC, 2007 causality costs
59
-CS -CS costs delay Disturbance d delay Benefits? ? ? benefits
delay Benefits? ? ? -CS benefits stabilize -CS causality costs
60
Physical implementation?
delay Disturbance - e=d-u d Physical implementation? Plant Sense/ Encode Control CS Abstract models of resource use Foundations, origins of noise dissipation amplification catalysis
61
Chandra, Buzi, and Doyle
62
CSTR, yeast extracts Experiments
K Nielsen, PG Sorensen, F Hynne, H-G Busse. Sustained oscillations in glycolysis: an experimental and theoretical study of chaotic and complex periodic behavior and of quenching of simple oscillations. Biophys Chem 72:49-62 (1998).
63
“Standard” Simulation
20 40 60 80 100 120 140 160 180 200 1 2 3 4 v=0.03 0.5 1.5 v=0.1 0.2 0.4 0.6 0.8 v=0.2 Figure S4. Simulation of two state model (S7.1) qualitatively recapitulates experimental observation from CSTR studies [5] and [12]. As the flow of material in/out of the system is increased, the system enters a limit cycle and then stabilizes again. For this simulation, we take q=a=Vm=1, k=0.2, g=1, u=0.01, h=2.5.
64
Why? Experiments Simulation
20 40 60 80 100 120 140 160 180 200 1 2 3 4 v=0.03 0.5 1.5 v=0.1 0.2 0.4 0.6 0.8 v=0.2 Experiments Simulation Why? Figure S4. Simulation of two state model (S7.1) qualitatively recapitulates experimental observation from CSTR studies [5] and [12]. As the flow of material in/out of the system is increased, the system enters a limit cycle and then stabilizes again. For this simulation, we take q=a=Vm=1, k=0.2, g=1, u=0.01, h=2.5.
65
Glycolytic “circuit” and oscillations
Most studied, persistent mystery in cell dynamics End of an old story (why oscillations) side effect of hard robustness/efficiency tradeoffs no purpose per se just needed a theorem Beginning of a new one robustness/efficiency tradeoffs complexity and architecture need more theorems and applications
66
x? y? autocatalytic? a? fragile? h? g? Robust
PFK? fragile? Tradeoffs? Hard limit? h? x? control? y? g? Robust =maintain energy charge w/fluctuating cell demand PK? Rest? rate k? robust? efficient? wasteful? Efficient=minimize metabolic overhead
67
z and p functions of enzyme complexity and amount
Theorem! z and p functions of enzyme complexity and amount Fragility simple enzyme complex enzyme Enzyme amount Savageaumics
68
almost Inside every cell ATP Macro-layers Precursors Catabolism
AA Enzymes Nucl. Macro-layers Building Blocks AA transl. Proteins ATP Ribosome RNA transc. xRNA Crosslayer autocatalysis RNAp DNA Repl. Gene DNAp
69
Energy Protein biosyn ATP Macro-layers Precursors Catabolism Enzymes
AA Enzymes Nucl. Macro-layers Building Blocks AA transl. Proteins ATP Protein biosyn Ribosome RNA transc. xRNA Crosslayer autocatalysis RNAp DNA Repl. Gene DNAp
70
Energy ATP Precursors Catabolism
71
x Metabolic flux ATP ATP Reaction 1 (“PFK”) energy Rest of cell
Minimal model Metabolic flux Reaction 1 (“PFK”) energy Rest of cell intermediate metabolite ATP ATP x Reaction 2 (“PK”) metabolic overhead Efficient
72
enzymes catalyze reactions
Reaction 1 (“PFK”) Rest of cell enzymes enzymes Protein biosyn Reaction 2 (“PK”) enzymes metabolic overhead enzyme amount Efficient
73
Fluorescence histogram (fluorescence vs
Fluorescence histogram (fluorescence vs. cell count) of GFP-tagged Glyceraldehyde-3-phosphate dehydrogenase (TDH3). Cells grown in ethanol has lower mean and median of fluorescence, and also higher variability.
74
1 2 3 10 10 10 Metabolic Overhead
75
Fragility highly variable g=0 is implausibly fragile g=0 g=1
10 g=0 1 Fragility g=1 g=0 is implausibly fragile -1 10 -1 1 10 10 10 Metabolic Overhead
76
autocatalytic feedback: energy
Reaction 1 (“PFK”) energy Rest of cell ATP enzymes Protein biosyn Reaction 2 (“PK”) enzymes enzymes metabolic overhead enzyme amount Efficient
77
x Metabolic flux ATP ATP energy Reaction 1 (“PFK”) Rest of cell
Inherently unstable ATP ATP x Reaction 2 (“PK”) metabolic overhead enzyme amount Efficient
78
x control feedback control ATP ATP Robust = Maintain energy
disturbance Reaction 1 (“PFK”) Fragile h energy control ATP ATP x Rest of cell g Reaction 2 (“PK”) Robust Robust = Maintain energy despite demand fluctuation
79
x control ATP ATP disturbance Reaction 1 (“PFK”) Fragile h energy
Rest of cell g Reaction 2 (“PK”) Robust metabolic overhead enzyme amount Efficient
80
Theorem! Fragile Robust metabolic overhead Efficient enzyme amount x
ATP Rest of cell energy x h g control Reaction 2 (“PK”) Reaction 1 (“PFK”) disturbance Fragile Robust metabolic overhead enzyme amount Efficient
81
Theorem! Fragile simple enzyme complex enzyme Robust
ATP Rest of cell energy x h g control Reaction 2 (“PK”) Reaction 1 (“PFK”) disturbance Fragile simple enzyme complex enzyme Robust metabolic overhead enzyme amount Efficient
82
Fragile complex enzyme Robust metabolic overhead Efficient
ATP Rest of cell energy x h g control Reaction 2 (“PK”) Reaction 1 (“PFK”) disturbance Fragile complex enzyme Robust metabolic overhead enzyme amount Efficient
83
y = WS WS h H [P W] y=ATP “weighed sensitivity” x ATP ATP control
disturbance Reaction 1 (“PFK”) h energy H [P W] x ATP ATP Rest of cell g Reaction 2 (“PK”) y=ATP control
84
Architecture Good architectures allow for effective tradeoffs fragile
Alternative systems with shared architecture “Conservation laws” wasteful
85
Theorem z and p are functions of enzyme complexity and amount
standard biochemistry models phenomenological first principles?
86
? Fragility hard limits General Rigorous First principle simple
complex ? Overhead, waste Domain specific Ad hoc Phenomenological Plugging in domain details
87
? Control Comms Fundamental multiscale physics Foundations, origins of
Wiener Comms Bode Shannon robust control Kalman Fundamental multiscale physics Foundations, origins of noise dissipation amplification catalysis General Rigorous First principle ? Carnot Boltzmann Heisenberg Physics
88
“New sciences” of complexity and networks
Alderson &Doyle, Contrasting Views of Complexity and Their Implications for Network-Centric Infrastructure, IEEE TRANS ON SMC, JULY 2010 Control Comms Complex networks Compute doesn’t work “New sciences” of complexity and networks edge of chaos, self-organized criticality, scale-free,… Stat physics Carnot Boltzmann Heisenberg Physics
89
Control Comms Complex networks Compute Stat physics Physics
Alderson &Doyle, Contrasting Views of Complexity and Their Implications for Network-Centric Infrastructure, IEEE TRANS ON SMC, JULY 2010 Control Comms Complex networks Compute doesn’t work Sandberg, Delvenne, & Doyle, On Lossless Approximations, the Fluctuation-Dissipation Theorem, and Limitations of Measurement, IEEE TRANS ON AC, FEBRUARY, 2011 Stat physics Carnot Boltzmann Heisenberg Physics
90
Control Comms Complex networks Compute “orthophysics” Stat physics,
Sandberg, Delvenne, & Doyle, On Lossless Approximations, the Fluctuation-Dissipation Theorem, and Limitations of Measurement, IEEE TRANS ON AC, FEBRUARY, 2011 Stat physics, fluids, QM Carnot Boltzmann Heisenberg Physics
91
Turbulence and drag? J. Fluid Mech (2010) Flow Streamlined
Laminar Flow Transition to Turbulence Increasing Drag, Fuel/Energy Use and Cost Turbulent Flow
92
Coherent structures and turbulent drag
Physics of Fluids (2011) y y Flow z x z x Coherent structures and turbulent drag high-speed region 3D coupling upflow Blunted turbulent velocity profile low speed streak downflow Turbulent Laminar
93
? Laminar Control? Turbulent Turbulent Laminar fragile robust
efficient wasteful
94
Foundations, origins of noise dissipation amplification catalysis
Control Wiener Comms Bode Shannon robust control Kalman Foundations, origins of noise dissipation amplification catalysis General Rigorous First principle Carnot Boltzmann Heisenberg Physics
95
Smart Antennas (Javad Lavaei)
Security, co-channel interference, power consumption Conventional Antenna Smart Antenna Smart Antennas 1) Multiple active elements: Easy to program Hard to implement 2) Multiple passive elements: Easy to implement Hard to program
96
Passively Controllable Smart (PCS) Antenna
PCS Antenna: One active element, reflectors and several parasitic elements This type of antenna is easy to program and easy to implement but “hard” to solve (e.g. 4 weeks offline computation). We solved the problem in 1 sec with huge improvement:
97
Decentralized control Partial bibliography (Lamperski)
Triaged today Great topic
98
In press, available online
Really fat tails How big is “big”?
99
“characteristic earthquake”???
Note: rare example (in science) involving power laws that isn’t obviously ridiculous smaller Persistent controversy Gutenberg-Richter log(rank) “characteristic earthquake”??? slope = -1 ? “bump” largest magnitude log(power)
102
Randomly sampled G-R magnitudes
Distribution of max event No bump still might look like a “bump”
103
Tail stays highly variable
Order statistics P( (k of n) > x) p=- dP/dx More data, but… Tail stays highly variable More samples
104
Fault traces and epicenters
105
155 faults Synthetic GR Mag of largest quake
Number of earthquakes per fault 155 faults Synthetic GR Number of earthquakes per fault Mag of largest quake
106
3 biggest quakes Mag of largest quake
107
3 biggest quakes p=.03 p=.01 p=.002
108
Magnitude binned by distance to faults
3.6
109
Randomly sampled G-R magnitudes
Distribution of max event No bump still might look like a “bump”
110
Los Padres National Forest
Truncated pareto model •P(>x) 3 10 Wildfires 2 10 1 10 LPNF data (decimated) 10 -1 1 2 3 10 10 10 10 10
113
Comparison of model n•P(>x) with cumulative raw data (decimated).
3 3 10 10 10 2 10 2 1 1 10 LPNF data (decimated) 10 10 10 10 -1 10 10 1 10 2 10 3 10 -2 10 -1 10 10 1 10 2
114
LPNF data (black) plus 4 pseudo-random samples from P(>x) (colors).
Comparison of variations in LPNF data versus that of pseudo-random samples. LPNF data (black) plus 4 pseudo-random samples from P(>x) (colors). LPNF data and pseudo-random samples have similar variations. 3 3 10 10 10 2 10 2 10 1 10 1 10 10 10 -1 10 10 1 10 2 10 3 10 -2 10 -1 10 10 1 10 2
115
MLE as WLS Exponential Pareto
116
MLE as WLS Exponential
117
Pareto Distribution α=1 n=100 K-S p-value: 0.34
118
Pareto Distribution MATLAB’s ksstest function with the null hypothesis Pareto alpha=1 α=1 n=100 p=0.34 2 4 6 8 10 10 10 10 10 x
119
α=1 n=100 p=0.34!!! makes no difference
MATLAB’s ksstest function with the null hypothesis Pareto alpha=1 10 2 Test Distribution Null Distribution α=1 n=100 p=0.34!!! Rank 1 10 makes no difference need weighted KS, but what weight? 10 2 4 6 8 10 10 10 10 10 x
120
Order statistics P( (k of n) > x) p=- dP/dx
But this is so easy and is (apparently) advocated by many statisticians. More samples
Similar presentations
© 2024 SlidePlayer.com. Inc.
All rights reserved.