Chess Review November 18, 2004 Berkeley, CA Hybrid Systems Theory Edited and Presented by Thomas A. Henzinger, Co-PI UC Berkeley.

Slides:

Advertisements

Similar presentations

From Graph Models to Game Models Tom Henzinger EPFL.

Advertisements

S4 S4 System Synthesis and Supervision, Scenarios Benoît Caillaud 20 March 2012.

Distributed Markov Chains P S Thiagarajan School of Computing, National University of Singapore Joint work with Madhavan Mukund, Sumit K Jha and Ratul.

This Segment: Computational game theory Lecture 1: Game representations, solution concepts and complexity Tuomas Sandholm Computer Science Department Carnegie.

Timed Automata.

COMP 553: Algorithmic Game Theory Fall 2014 Yang Cai Lecture 21.

Game Theoretical Insights in Strategic Patrolling: Model and Analysis Nicola Gatti – DEI, Politecnico di Milano, Piazza Leonardo.

Energy and Mean-Payoff Parity Markov Decision Processes Laurent Doyen LSV, ENS Cachan & CNRS Krishnendu Chatterjee IST Austria MFCS 2011.

Game Theory: introduction and applications to computer networks Game Theory: introduction and applications to computer networks Zero-Sum Games (follow-up)

Evolutionary Game Algorithm for continuous parameter optimization Alireza Mirian.

Interface-based design Philippe Giabbanelli CMPT 894 – Spring 2008.

Krishnendu Chatterjee1 Partial-information Games with Reachability Objectives Krishnendu Chatterjee Formal Methods for Robotics and Automation July 15,

Randomness for Free Laurent Doyen LSV, ENS Cachan & CNRS joint work with Krishnendu Chatterjee, Hugo Gimbert, Tom Henzinger.

An Introduction to Game Theory Part I: Strategic Games

IN SEARCH OF VALUE EQUILIBRIA By Christopher Kleven & Dustin Richwine xkcd.com.

IN SEARCH OF VALUE EQUILIBRIA By Christopher Kleven & Dustin Richwine xkcd.com.

Convergent Learning in Unknown Graphical Games Dr Archie Chapman, Dr David Leslie, Dr Alex Rogers and Prof Nick Jennings School of Mathematics, University.

1 Verification and Synthesis of Hybrid Systems Thao Dang October 10, 2000.

Fault-Tolerant Real-Time Networks Tom Henzinger UC Berkeley MURI Kick-off Workshop Berkeley, May 2000.

Discounting the Future in Systems Theory Chess Review May 11, 2005 Berkeley, CA Luca de Alfaro, UC Santa Cruz Tom Henzinger, UC Berkeley Rupak Majumdar,

Quantitative Verification Arindam Chakrabarti * Krishnendu Chatterjee * Thomas A. Henzinger * Orna Kupferman ** Rupak Majumdar *** * UC Berkeley ** Hebrew.

Systems Engineering for Automating V&V of Dependable Systems John S. Baras Institute for Systems Research University of Maryland College Park

Convertibility Verification and Converter Synthesis: Two Faces of the Same Coin Jie-Hong Jiang EE249 Discussion 11/21/2002 Passerone et al., ICCAD ’ 02.

Stochastic Zero-sum and Nonzero-sum  -regular Games A Survey of Results Krishnendu Chatterjee Chess Review May 11, 2005.

Games, Times, and Probabilities: Value Iteration in Verification and Control Krishnendu Chatterjee Tom Henzinger.

Models and Theory of Computation (MTC) EPFL Dirk Beyer, Jasmin Fisher, Nir Piterman Simon Kramer: Logic for cryptography Marc Schaub: Models for biological.

INSTITUTO DE SISTEMAS E ROBÓTICA Minimax Value Iteration Applied to Robotic Soccer Gonçalo Neto Institute for Systems and Robotics Instituto Superior Técnico.

Autonomous Target Assignment: A Game Theoretical Formulation Gurdal Arslan & Jeff Shamma Mechanical and Aerospace Engineering UCLA AFOSR / MURI.

Matrix Games Mahesh Arumugam Borzoo Bonakdarpour Ali Ebnenasir CSE 960: Selected Topics in Algorithms and Complexity Instructor: Dr. Torng.

XYZ 6/18/2015 MIT Brain and Cognitive Sciences Convergence Analysis of Reinforcement Learning Agents Srinivas Turaga th March, 2004.

Stochastic Games Games played on graphs with stochastic transitions Markov decision processes Games against nature Turn-based games Games against adversary.

Outline MDP (brief) –Background –Learning MDP Q learning Game theory (brief) –Background Markov games (2-player) –Background –Learning Markov games Littman’s.

An Introduction to Game Theory Part III: Strictly Competitive Games Bernhard Nebel.

Rich Interface Theories for Component-based Design Dirk Beyer ┼, Arindam Chakrabarti *, Luca de Alfaro **, Thomas A Henzinger * ┼, Marcin Jurdziński *,

Nash Q-Learning for General-Sum Stochastic Games Hu & Wellman March 6 th, 2006 CS286r Presented by Ilan Lobel.

Chess Review October 4, 2006 Alexandria, VA Edited and presented by Hybrid Systems: Theoretical Contributions Part I Shankar Sastry UC Berkeley.

Chess Review November 21, 2005 Berkeley, CA Edited and presented by Advances in Hybrid System Theory: Overview Claire J. Tomlin UC Berkeley.

NSF Foundations of Hybrid and Embedded Software Systems UC Berkeley: Chess Vanderbilt University: ISIS University of Memphis: MSI Hybrid Systems: From.

Designing Predictable and Robust Systems Tom Henzinger UC Berkeley and EPFL.

Report WG1 Software-Intensive Systems and New Computing Paradigms Cannes November 12-14, 2008 WG Leader: Martin Wirsing WG Depu ty Leaders: Jean-Pierre.

Games in the normal form- An application: “An Economic Theory of Democracy” Carl Henrik Knutsen 5/

Game Theory April 9, Prisoner’s Dilemma  One-shot, simultaneous game  Nash Equilibrium (individually rational strategies) is not Pareto Optimal.

Stochastic Games Krishnendu Chatterjee CS 294 Game Theory.

Chess Review May 10, 2004 Berkeley, CA Rich Interface Theories for Component-based Design Arindam Chakrabarti Luca de Alfaro Thomas A. Henzinger Marcin.

NSF Foundations of Hybrid and Embedded Software Systems UC Berkeley: Chess Vanderbilt University: ISIS University of Memphis: MSI Program Review May 10,

Abstract Verification is traditionally done by determining the truth of a temporal formula (the specification) with respect to a timed transition system.

6.896: Topics in Algorithmic Game Theory Spring 2010 Constantinos Daskalakis vol. 1:

Minimax strategies, Nash equilibria, correlated equilibria Vincent Conitzer

01/16/2002 Reliable Query Reporting Project Participants: Rajgopal Kannan S. S. Iyengar Sudipta Sarangi Y. Rachakonda (Graduate Student) Sensor Networking.

MAKING COMPLEX DEClSlONS

Derivative Action Learning in Games Review of: J. Shamma and G. Arslan, “Dynamic Fictitious Play, Dynamic Gradient Play, and Distributed Convergence to.

Energy Parity Games Laurent Doyen LSV, ENS Cachan & CNRS Krishnendu Chatterjee IST Austria.

Stochastic Algorithms Some of the fastest known algorithms for certain tasks rely on chance Stochastic/Randomized Algorithms Two common variations – Monte.

1 S ystems Analysis Laboratory Helsinki University of Technology Kai Virtanen, Janne Karelahti, Tuomas Raivio, and Raimo P. Hämäläinen Systems Analysis.

Uri Zwick Tel Aviv University Simple Stochastic Games Mean Payoff Games Parity Games TexPoint fonts used in EMF. Read the TexPoint manual before you delete.

Epistemic Strategies and Games on Concurrent Processes Prakash Panangaden: Oxford University (on leave from McGill University). Joint work with Sophia.

1 S ystems Analysis Laboratory Helsinki University of Technology Kai Virtanen, Tuomas Raivio and Raimo P. Hämäläinen Systems Analysis Laboratory Helsinki.

Games with Secure Equilibria Krishnendu Chatterjee (Berkeley) Thomas A. Henzinger (EPFL) Marcin Jurdzinski (Warwick)

1 S ystems Analysis Laboratory Helsinki University of Technology Kai Virtanen, Janne Karelahti, Tuomas Raivio, and Raimo P. Hämäläinen Systems Analysis.

Equilibrium transitions in stochastic evolutionary games Dresden, ECCS’07 Jacek Miękisz Institute of Applied Mathematics University of Warsaw.

Vincent Conitzer CPS Learning in games Vincent Conitzer

DEPENDABILITY ANALYSIS (towards Networked Information Systems) Ester Ciancamerla, Michele Minichino ENEA {ciancamerlae, In.

Chapter 5 Adversarial Search. 5.1 Games Why Study Game Playing? Games allow us to experiment with easier versions of real-world situations Hostile agents.

Topics  Direct Predicate Characterization as an evaluation method.  Implementation and Testing of the Approach.  Conclusions and Future Work.

Sub-fields of computer science. Sub-fields of computer science.

Stochastic -Regular Games

High-Level Abstraction of Concurrent Finite Automata

Kai Virtanen, Janne Karelahti, Tuomas Raivio, and Raimo P. Hämäläinen

Quantitative Modeling, Verification, and Synthesis

Collaboration in Repeated Games

Presentation transcript:

Chess Review November 18, 2004 Berkeley, CA Hybrid Systems Theory Edited and Presented by Thomas A. Henzinger, Co-PI UC Berkeley

Chess Review, November 18, Formal Foundation for Embedded Systems needs to combine ComputationPhysicality + Theories of -composition & hierarchy -computability & complexity B Theories of -robustness & approximation -probabilities & discounting R

Chess Review, November 18, Continuous Dynamical Systems State space:R n Dynamics:initial condition + differential equations Room temperature:x(0) = x 0 x’(t) = -K·x(t) x t x0x0 Analytic complexity.

Chess Review, November 18, Discrete Transition Systems State space:B m Dynamics:initial condition + transition relation Heater: heat t off on offon Combinatorial complexity.

Chess Review, November 18, Hybrid Automata State space:B m  R n Dynamics:initial condition + transition relation + differential equations Thermostat: t off on x0x0 off x’ = -K·x on x’ = K·(H-x) x  l x  u x  U x  L

Chess Review, November 18, Four Problems with Hybrid Automata 1Robustness 2Uncertainty 3Compositionality 4Computationality

Chess Review, November 18, Safe Hybrid Automaton x = 3 The Robustness Issue

Chess Review, November 18, Unsafe Slightly Perturbed Hybrid Automaton x = 3+  The Robustness Issue

Chess Review, November 18, value(Model,Property): States  B value(Model,Property): States  R Robust Hybrid Automata Semantics: de Alfaro, H, Majumdar [ICALP 03] Computation: de Alfaro, Faella, H, Majumdar, Stoelinga [TACAS 04] Metrics on models: Chatterjee et al. [submitted]

Chess Review, November 18, acb Boolean-valued Reachability   c … True or False T (F Ç  pre(T)) = TT

Chess Review, November 18, acb Real-valued Reachability   c … True or False T (F Ç  pre(T)) = TT 2   c … between 0 and 1 1max(0, ¢  pre(1)) = discount factor 0 < < 1

Chess Review, November 18, Continuity Theorem: If discountedBisimilarity(m 1,m 2 ) > 1 - , then |discountedValue(m 1,p) - discountedValue(m 2,p)| < f(  ). Further Advantages of Discounting: -approximability because of geometric convergence (avoids non-termination of verification algorithms) -applies also to probabilistic systems and to games (enables reasoning under uncertainty, and control) Robust Hybrid Automata

Chess Review, November 18, Four Problems with Hybrid Automata 1Robustness 2Uncertainty 3Compositionality 4Computationality

Chess Review, November 18, The Uncertainty Issue Hybrid Automaton A 0 < x < 2 Hybrid Automaton B 1 < y < 3 ab

Chess Review, November 18, The Uncertainty Issue Composite Automaton A||B b a a,b more likely less likely impossible

Chess Review, November 18, acb 1,12,21,12,2 2,11,22,11,2 1,11,22,21,11,22,2 2,12,1 Concurrent Games player "left" player "right" -for modeling component-based systems (“interfaces”) -for strategy synthesis (“control”)

Chess Review, November 18, acb 1,1 2,2 2,1 1,2 1,1 1,2 2,2 2,1  left  right  c … player "left" has a deterministic strategy to reach c Concurrent Games (  X) (c   left  right pre(X))

Chess Review, November 18, acb 1,1 2,2 2,1 1,2 1,1 1,2 2,2 2,1  left  right  c … player "left" has a deterministic strategy to reach c  left  right  c … player "left" has a randomized strategy to reach c Concurrent Games Pr(1): 0.5 Pr(2): 0.5 (  X) (c   left  right pre(X))

Chess Review, November 18, acb left right a: 0.6 b: 0.4 a: 0.1 b: 0.9 a: 0.5 b: 0.5 a: 0.2 b: left right a: 0.0 c: 1.0 a: 0.7 b: 0.3 a: 0.0 c: 1.0 a: 0.0 b: 1.0 Probability with which player "left" can reach c ? Stochastic Games

Chess Review, November 18, acb left right a: 0.6 b: 0.4 a: 0.1 b: 0.9 a: 0.5 b: 0.5 a: 0.2 b: left right a: 0.0 c: 1.0 a: 0.7 b: 0.3 a: 0.0 c: 1.0 a: 0.0 b: 1.0 Probability with which player "left" can reach c ? Stochastic Games (  X) max(c,  left  right pre(X))

Chess Review, November 18, acb left right a: 0.6 b: 0.4 a: 0.1 b: 0.9 a: 0.5 b: 0.5 a: 0.2 b: left right a: 0.0 c: 1.0 a: 0.7 b: 0.3 a: 0.0 c: 1.0 a: 0.0 b: 1.0 Probability with which player "left" can reach c ? Stochastic Games (  X) max(c,  left  right pre(X))

Chess Review, November 18, acb left right a: 0.6 b: 0.4 a: 0.1 b: 0.9 a: 0.5 b: 0.5 a: 0.2 b: left right a: 0.0 c: 1.0 a: 0.7 b: 0.3 a: 0.0 c: 1.0 a: 0.0 b: 1.0 Probability with which player "left" can reach c ? Stochastic Games 11 1 Limit gives correct answer: de Alfaro, Majumdar [JCSS 04] coNP Å NP computation: Chatterjee, de Alfaro, H [submitted]

Chess Review, November 18, Four Problems with Hybrid Automata 1Robustness 2Uncertainty 3Compositionality 4Computationality

Chess Review, November 18, Model Requirements Resources Verification Implementation Environment automatic (model checking) automatic (compilation) The Compositionality Issue

Chess Review, November 18, Component Requirements Resources Verification Implementation Component The Compositionality Issue Composition no change necessary

Chess Review, November 18, Component Requirements Resources Verification Implementation Component The Compositionality Issue Composition no change necessary (time, fault tolerance, etc.)

Chess Review, November 18, Component Requirements Resources Verification Implementation Component The Compositionality Issue Composition no change necessary (time, fault tolerance, etc.) Agent algebras. Interface theories. Virtual machines.

Chess Review, November 18, Consider hybrid system made up of interacting distributed subsystems: Embedded Controller Physical Process … Embedded Controller Physical Process Subsystem 1 Subsystem N Logical Interaction Physical Interaction …  Physical subsystems coupled through a backbone  Each unit includes ECDs that implement the control, monitoring, and fault diagnosis tasks  Subsystem interactions at two levels:  physical – energy-based  logical – information based, facilitated by LANs Levels are not independent. Question: How does one systematically model the interactions between the subsystems efficiently while avoiding the computational complexity of generating global hybrid models? Implications: reachability analysis, design, control, and fault diagnosis Heterogeneous Compositional Modeling

Chess Review, November 18, Four Problems with Hybrid Automata 1Robustness 2Uncertainty 3Compositionality 4Computationality

Chess Review, November 18, The Computationality Issue system control, initial state Find reach set of all states that can be reached at time t starting in at t 0 using open loop control u(t). Reach Set Computation:

Chess Review, November 18, Ellipsoidal Toolbox Calculation of reach sets using ellipsoidal approximation algorithms Visualization of their 3D projections

Chess Review, November 18, Putting It All Together 1Robustness 2Uncertainty 3Compositionality 4Computationality

Chess Review, November 18, Classification of 2-Player Games Zero-sum games: complementary payoffs. Non-zero-sum games: arbitrary payoffs. 1,-10,0 2,-2-1,1 3,11,0 4,23,2

Chess Review, November 18, Classical Notion of Rationality Nash equilibrium: none of the players gains by deviation. 3,11,0 4,23,2 (row, column)

Chess Review, November 18, Classical Notion of Rationality Nash equilibrium: none of the players gains by deviation. 3,11,0 4,23,2 (row, column)

Chess Review, November 18, New Notion of Rationality Nash equilibrium: none of the players gains by deviation. Secure equilibrium: none hurts the opponent by deviation. 3,11,0 4,23,2 (row, column)

Chess Review, November 18, Secure Equilibria Natural notion of rationality for component systems: –First, a component tries to meet its spec. –Second, a component may obstruct the other components. For Borel specs, there is always unique maximal secure equilibrium.

Chess Review, November 18, Synthesis: - Zero-sum game controller versus plant. - Control against all plant behaviors. Verification: - Non-zero-sum specs for components. - Components may behave adversarially, but without threatening their own specs. Borel Games on State Spaces

Chess Review, November 18, Borel Games on State Spaces Zero-sum games: –Complementary objectives:  2 = :  1. –Possible payoff profiles (1,0) and (0,1). Non-zero-sum games: –Arbitrary objectives  1,  2. –Possible payoff profiles (1,1), (1,0), (0,1), and (0,0).

Chess Review, November 18, Zero-Sum Borel Games Winning: - Winning-1 states s: ( 9  ) ( 8  )  ,  (s) 2  1. - Winning-2 states s: ( 9  ) ( 8  )  ,  (s) 2  2. Determinacy: –Every state is winning-1 or winning-2. –Borel determinacy [Martin 75]. –Memoryless determinacy for parity games [Emerson/Jutla 91]. (1,0)(0,1)

Chess Review, November 18, Secure Equilibria Secure strategy profile ( ,  ) at state s: ( 8  ’) ( v 1 ,  ’ (s) < v 1 ,  (s) ) v 2 ,  ’ (s) < v 2 ,  (s) ) ( 8  ’) ( v 2  ’,  (s) < v 2 ,  (s) ) v 1  ’,  (s) < v 1 ,  (s) ) A secure profile ( ,  ) is a contract: if the player-1 deviates to lower player-2’s payoff, her own payoff decreases as well, and vice versa. Secure equilibrium: secure strategy profile that is also a Nash equilibrium.

Chess Review, November 18, State Space Partition

Chess Review, November 18, W 10 hh1ii (  1 Æ :  2 ) hh2ii ( :  1 Ç  2 ) Computing the Partition

Chess Review, November 18, Computing the Partition W 10 hh1ii (  1 Æ :  2 ) hh2ii (  1 )  2 ) W 01 hh2ii (  2 Æ :  1 ) hh1ii (  2 )  1 )

Chess Review, November 18, Computing the Partition W 10 hh1ii (  1 Æ :  2 ) hh2ii (  1 )  2 ) W 01 hh2ii (  2 Æ :  1 ) hh1ii (  2 )  1 ) hh1ii  1 U1U1

Chess Review, November 18, Computing the Partition W 10 hh1ii (  1 Æ :  2 ) hh2ii (  1 )  2 ) W 01 hh2ii (  2 Æ :  1 ) hh1ii (  2 )  1 ) hh1ii  1 hh2ii  2 U1U1 U2U2

Chess Review, November 18, Computing the Partition W 10 hh1ii (  1 Æ :  2 ) hh2ii (  1 )  2 ) W 01 hh2ii (  2 Æ :  1 ) hh1ii (  2 )  1 ) hh1ii  1 hh2ii  2 Threat strategies  T,  T hh2ii :  1 hh1ii :  2 U1U1 U2U2

Chess Review, November 18, Computing the Partition W 10 hh1ii (  1 Æ :  2 ) hh2ii (  1 )  2 ) W 01 hh2ii (  2 Æ :  1 ) hh1ii (  2 )  1 ) hh1ii  1 hh2ii  2 Threat strategies  T,  T hh2ii :  1 hh1ii :  2 hh1,2ii (  1 Æ  2 ) Cooperation strategies  C,  C U1U1 U2U2

Chess Review, November 18, Computing the Partition W 10 hh1ii (  1 Æ :  2 ) W 01 hh2ii (  2 Æ :  1 ) hh1ii  1 hh2ii  2 hh1,2ii (  1 Æ  2 ) U1U1 U2U2 W00W00

Chess Review, November 18, Generalization of Determinacy W 10 W 01 W 11 W 00 W1W1 W2W2 Zero-sum games:  2 = :  1 Non-zero-sum games:  1,  2

Chess Review, November 18, P 1 2 W 1 (  1 ) P 2 2 W 2 (  2 )  1 Æ  2 )  P 1 ||P 2 ²  Application: Compositional Verification

Chess Review, November 18, P 1 2 W 1 (  1 ) P 2 2 W 2 (  2 )  1 Æ  2 )  P 1 ||P 2 ²  Application: Compositional Verification P 1 2 (W 10 [ W 11 ) (  1 ) P 2 2 (W 01 [ W 11 ) (  2 )  1 Æ  2 )  P 1 ||P 2 ²  W 1 ½ W 10 [ W 11 W 2 ½ W 01 [ W 11 An assume/guarantee rule.

Chess Review, November 18, Related In-Depth Talks Roberto Passerone (11:50 am): -semantics of hybrid systems Aaron Ames (12:10 pm): -stochastic approximation of hybrid systems -a categorical theory of hybrid systems

Chess Review, November 18, Related Posters Robust Hybrid Systems: Blowing up Hybrid Systems (Aaron Ames) Quantitative Verification (Vinayak Prabhu) Compositional Hybrid Systems: Rich Interface Theories (Arindam Chakrabarti) Stochastic Hybrid Systems: Stochastic Games (Krishnendu Chatterjee) Computational Hybrid Systems: Computation of Reach Sets (Alex Kurzhansky)