Chess Review October 4, 2006 Alexandria, VA Edited and presented by Hybrid Systems: Theoretical Contributions Part I Shankar Sastry UC Berkeley

ITR Review, Oct. 4, 2006"Hybrid Systems Theory: I", S. Sastry2 Broad Theory Contributions: Samples Sastry’s group: Defined and set the agenda of the following sub-fields –Stochastic Hybrid Systems – Category Theoretic View of Hybrid Systems, –State Estimation of Partially Observable Hybrid Systems Tomlin’s group: Developed new mathematics for –Safe set calculations and approximations, –Estimation of hybrid systems Sangiovanni’s group defined –“Intersection based composition”-model as common fabric for metamodeling, –Contracts and contract algebra + refinement relation for assumptions/promises-based design in metamodel

ITR Review, Oct. 4, 2006"Hybrid Systems Theory: I", S. Sastry3 Quantitative Verification for Discrete- Time Stochastic Hybrid Systems (DTSHS) Stochastic hybrid systems (SHS) can model uncertain dynamics and stochastic interactions that arise in many systems Quantitative verification problem: –What is the probability with which the system can reach a set during some finite time horizon? –(If possible), select a control input to ensure that the system remains outside the set with sufficiently high probability –When the set is unsafe, find the maximal safe sets corresponding to different safety levels [Abate, Amin, Prandini, Lygeros, Sastry] HSCC 2006

ITR Review, Oct. 4, 2006"Hybrid Systems Theory: I", S. Sastry4 Qualitative vs. Quantitative Verification System is safeSystem is unsafe System is safe with probability 1.0System is unsafe with probability ε Qualitative Verification Quantitative Verification

ITR Review, Oct. 4, 2006"Hybrid Systems Theory: I", S. Sastry5 Discrete-Time Stochastic Hybrid Systems

ITR Review, Oct. 4, 2006"Hybrid Systems Theory: I", S. Sastry6 Entities

ITR Review, Oct. 4, 2006"Hybrid Systems Theory: I", S. Sastry7 Definition of Reach Probability

ITR Review, Oct. 4, 2006"Hybrid Systems Theory: I", S. Sastry8 Reachability as Safety Specification

ITR Review, Oct. 4, 2006"Hybrid Systems Theory: I", S. Sastry9 Computation of Optimal Reach Probability

ITR Review, Oct. 4, 2006"Hybrid Systems Theory: I", S. Sastry10 Room Heating Benchmark Temperature sensors Room 1 Room 2 Heater Two Room One Heater Example Temperature in two rooms is controlled by one heater. Safe set for both rooms is 20 – 25 ( 0 F) Goal is to keep the temperatures within corresponding safe sets with a high probability SHS model –Two continuous states: –Three modes: OFF, ON (Room 1), ON (Room 2) – Continuous evolution in mode ON (Room 1) –Mode switches defined by controlled Markov chain with seven discrete actions: (Do Nothing, Rm 1->Rm2, Rm 2-Rm 1, Rm 1-> Rm 3, Rm 3->Rm1, Rm 2-Rm 3, Rm 3-> Rm 2)

ITR Review, Oct. 4, 2006"Hybrid Systems Theory: I", S. Sastry11 Probabilistic Maximal Safe Sets for Room Heating Benchmark (for initial mode OFF) Temperature in Room 1 Temperature in Room 2 Starting from this initial condition in OFF mode and following optimal control law, it is guaranteed that system will remain in the safe set (20,25)×(20,25) 0 F with probability at least 0.9 for 150 minutes Note: The spatial discretization is F, temporal discretization is 1 min and time horizon is 150 minutes

ITR Review, Oct. 4, 2006"Hybrid Systems Theory: I", S. Sastry12 Optimal Control Actions for Room Heating Benchmark (for initial mode OFF)

ITR Review, Oct. 4, 2006"Hybrid Systems Theory: I", S. Sastry13 More Results Alternative interpretation –Problem of keeping the state of DTSHS outside some pre-specified “unsafe” set by selecting suitable feedback control law can be formulated as a optimal control problem with “max”-cost function –Value functions for “max”-cost case can be expressed in terms of value functions for “multiplicative”-cost case Time varying safe set specification can be incorporated within the current framework Extension to infinite-horizon setting and convergence of optimal control law to stationary policy is also addressed [Abate, Amin, Prandini, Lygeros, Sastry] CDC2006

ITR Review, Oct. 4, 2006"Hybrid Systems Theory: I", S. Sastry14 Future Work Within the current setup –Sufficiency of Markov policies –Randomized policies, partial information case –Interpretation as killed Markov chain –Distributed dynamic programming techniques Extensions to continuous time setup –Discrete time controlled SHS as stochastic approx. of general continuous time controlled SHS Embedding performance in the problem setup Extensions to game theoretic setting

Chess Review October 4, 2006 Alexandria, VA Edited and presented by A Categorical Theory of Hybrid Systems Aaron Ames

ITR Review, Oct. 4, 2006"Hybrid Systems Theory: I", S. Sastry16 Motivation and Goal Hybrid systems represent a great increase in complexity over their continuous and discrete counterparts A new and more sophisticated theory is needed to describe these systems: categorical hybrid systems theory –Reformulates hybrid systems categorically so that they can be more easily reasoned about –Unifies, but clearly separates, the discrete and continuous components of a hybrid system –Arbitrary non-hybrid objects can be generalized to a hybrid setting –Novel results can be established

ITR Review, Oct. 4, 2006"Hybrid Systems Theory: I", S. Sastry17 Hybrid Category Theory: Framework One begins with: – A collection of “non-hybrid” mathematical objects – A notion of how these objects are related to one another (morphisms between the objects) Example: vector spaces, manifolds Therefore, the non-hybrid objects of interest form a category, Example: The objects being considered can be “hybridized” by considering a small category (or “graph”) together with a functor (or “function”): – is the “discrete” component of the hybrid system – is the “continuous” component Example: hybrid vector space hybrid manifold T T = V ec t ; T = M an ; T D D S : D ! T S : D ! V ec t, S : D ! M an.

ITR Review, Oct. 4, 2006"Hybrid Systems Theory: I", S. Sastry18 Applications The categorical framework for hybrid systems has been applied to: –Geometric Reduction Generalizing to a hybrid setting –Bipedal robotic walkers Constructing control laws that result in walking in three-dimensions –Zeno detection Sufficient conditions for the existence of Zeno behavior

ITR Review, Oct. 4, 2006"Hybrid Systems Theory: I", S. Sastry19 Applications –Geometric Reduction Generalizing to a hybrid setting –Bipedal robotic walkers Constructing control laws that result in walking in three-dimensions –Zeno detection Sufficient conditions for the existence of Zeno behavior

ITR Review, Oct. 4, 2006"Hybrid Systems Theory: I", S. Sastry20 Hybrid Reduction: Motivation Reduction decreases the dimensionality of a system with symmetries –Circumvents the “curse of dimensionality” –Aids in the design, analysis and control of systems –Hybrid systems are hard—reduction is more important!

ITR Review, Oct. 4, 2006"Hybrid Systems Theory: I", S. Sastry21 Hybrid Reduction: Motivation Problem: –There are a multitude of mathematical objects needed to carry out classical (continuous) reduction –How can we possibly generalization? Using the notion of a hybrid object over a category, all of these objects can be easily hybridized Reduction can be generalized to a hybrid setting

ITR Review, Oct. 4, 2006"Hybrid Systems Theory: I", S. Sastry22 Hybrid Reduction Theorem

ITR Review, Oct. 4, 2006"Hybrid Systems Theory: I", S. Sastry23 Applications –Geometric Reduction Generalizing to a hybrid setting –Bipedal robotic walkers Constructing control laws that result in walking in three-dimensions –Zeno detection Sufficient conditions for the existence of Zeno behavior

ITR Review, Oct. 4, 2006"Hybrid Systems Theory: I", S. Sastry24 Bipedal Robots and Geometric Reduction Bipedal robotic walkers are naturally modeled as hybrid systems The hybrid geometric reduction theorem is used to construct walking gaits in three dimensions given walking gaits in two dimensions

ITR Review, Oct. 4, 2006"Hybrid Systems Theory: I", S. Sastry25 Goal

ITR Review, Oct. 4, 2006"Hybrid Systems Theory: I", S. Sastry26 How to Walk in Four Easy Steps

ITR Review, Oct. 4, 2006"Hybrid Systems Theory: I", S. Sastry27 Simulations

ITR Review, Oct. 4, 2006"Hybrid Systems Theory: I", S. Sastry28 Applications –Geometric Reduction Generalizing to a hybrid setting –Bipedal robotic walkers Constructing control laws that result in walking in three-dimensions –Zeno detection Sufficient conditions for the existence of Zeno behavior

ITR Review, Oct. 4, 2006"Hybrid Systems Theory: I", S. Sastry29 Zeno Behavior and Mechanical Systems Mechanical systems undergoing impacts are naturally modeled as hybrid systems –The convergent behavior of these systems is often of interest –This convergence may not be to ``classical'' notions of equilibrium points –Even so, the convergence can be important –Simulating these systems may not be possible due to the relationship between Zeno equilibria and Zeno behavior.

ITR Review, Oct. 4, 2006"Hybrid Systems Theory: I", S. Sastry30 Zeno Behavior at Work Zeno behavior is famous for its ability to halt simulations To prevent this outcome: –A priori conditions on the existence of Zeno behavior are needed –Noticeable lack of such conditions

ITR Review, Oct. 4, 2006"Hybrid Systems Theory: I", S. Sastry31 Zeno Equilibria Hybrid models admit a kind of Equilibria that is not found in continuous or discrete dynamical systems: Zeno Equilibria. –A collection of points invariant under the discrete dynamics –Can be stable in many cases of interest. –The stability of Zeno equilibria implies the existence of Zeno behavior.

ITR Review, Oct. 4, 2006"Hybrid Systems Theory: I", S. Sastry32 Overview of Main Result The categorical approach to hybrid systems allows us to decompose the study of Zeno equilibria into two steps: 1.We identify a sufficiently rich, yet simple, class of hybrid systems that display the desired stability properties: first quadrant hybrid systems 2.We relate the stability of general hybrid systems to the stability of these systems through a special class of hybrid morphisms: hybrid Lyapunov functions

ITR Review, Oct. 4, 2006"Hybrid Systems Theory: I", S. Sastry33 Some closing thoughts Key new areas of research initiated Some important new results Additional theory needed especially for networked embedded systems