Solving Timed Games with Variable Observations: Proof of Concept Peter Bulychev Franck Cassez Alexandre David Kim G. Larsen Jean-François Raskin Pierre-Alain Reynier

GASICS Workshop2 Timed Game Automata Timed Game Automata is a Timed Automata where transitions are split into controllable and uncontrollable We support safety objectives: control: AG (not Bad) Memoryless strategy: state  action UPPAAL Tiga can be used to solve safety timed games a b

GASICS Workshop3 Timed Game Automata x≤1: a True: DELAY Strategy x≤1: b True: DELAY control: AG (not Bad) a b

GASICS Workshop4 Controller synthesis with partial observation Consider that controller doesn’t have full information about the current state of a system Observation is a valuation of a finite number of state-based boolean predicates (sensors) We allow predicates of the form: (L1 or L2 or L3) and (1≤x<2) Controller makes its decisions based on history of the observations seen so far Controller sees only changes on observations => stuttering-invariant strategy

GASICS Workshop5 Controller synthesis with partial observation: the algorithm Partition the state-space w.r.t. values of the predicates. Predicates p 1, p 2 Losing is observable. p1p2p1p2  p 1  p 2 p 1  p 2 p1p2p1p2 LOSING a a a b DELAY b

Running example (LH boxes) GASICS Workshop6 Possible sets of observations: {H, L} {H, L, y≥3} control: AG (not Bad) {y ≥ 1} {H, L, y≥5} full information {H, L, y≥1} EJECT RESET

GASICS Workshop7 Controller synthesis with partial observation: the algorithm Partition the state-space w.r.t. observations. Observations O1 O2 O3. Winning/losing is observable. Algorithm, described in F. Cassez et al., 2007: Symbolic On-the-fly Subset construction-based Implemented in UPPAAL Tiga

Running example (LH boxes) GASICS Workshop8 {} control: AG (not Bad) Available observations: {H, L, y ≥ 5} {H} {} DELAY {y ≥ 5} DELAY EJECT {}{y ≥ 5} DELAY {L} {} DELAY {y ≥ 5} DELAY EJECT E0,x==y==0 H,x==y==0 E1 \/ E2,x==y==0E1 \/ E2,x==y==5 RESET E1 \/ E2,x==5, y==0 E1 \/ E2,x==10, y==5 H,x==y==0 E3 \/ E4,x==y==0E3 \/ E4,x==y==5

Problem statement Assume a finite set of available sensors and each sensor has some cost We want to synthesize a controller that will achieve its goal by using a set of sensors with a minimal cost Input: Timed Game Automata A Safety propertyφ A set of predicates Pred = {p 1, …, p n } Cost function ω = {p 1 ->c 1, …, p n ->c n } Goal: To find a set of predicates P with a minimal total cost such that A,P|=φ is true GASICS Workshop9

Basic algorithm Consider a lattice of all possible predicates sets GASICS Workshop10 {φ}{φ} {φ} U Pred

Basic algorithm 1. Check if φ is controllable on A with full information GASICS Workshop11 {φ}{φ} {φ} U Pred Full information

Basic algorithm 1. Check if φ is controllable on A with full information 2. Check A,P|=φ for some set of predicates P GASICS Workshop12 {φ}{φ} {φ} U Pred P

Basic algorithm 1. Check if φ is controllable on A with full information 2. Check A,P|=φ for some set of predicates P 3. If A,P|=φ is true, then we remove from further consideration all sets P’ s.t. P ⊆ P’ GASICS Workshop13 {φ}{φ} {φ} U Pred P

Basic algorithm GASICS Workshop14 {φ}{φ} {φ} U Pred 1. Check if φ is controllable on A with full information 2. Check A,P|=φ for some set of predicates P 3. If A,P|=φ is true, then we remove from further consideration all sets P’ s.t. P ⊆ P’ remove from further consideration all sets P’ s.t. ω(P’) ≥ ω(P) P

Basic algorithm 1. Check if φ is controllable on A with full information 2. Check A,P|=φ for some set of predicates P 3. If A,P|=φ is true, then we remove from further consideration all sets P’ s.t. P ⊆ P’ remove from further consideration all sets P’ s.t. ω(P’) ≥ ω(P) 4. Otherwise, we remove from further consideration all sets P’ s.t. P’ ⊆ P GASICS Workshop15 {φ}{φ} {φ} U Pred

Basic algorithm The set of possible observation sets is finite, so the algorithm will converge GASICS Workshop16 {φ}{φ} {φ} U Pred

Basic algorithm Optimizations: Which exploration strategy to use? Random Top-bottom Bottom-top Midpoint What information to reuse? Losing states from below Winning states from above State space from below GASICS Workshop17 {φ}{φ} {φ} U Pred

Basic algorithm Optimizations: Which exploration strategy to use? Random Top-bottom Bottom-top Midpoint What information to reuse? Losing states from below Winning states from above State space from below GASICS Workshop18 {φ}{φ} {φ} U Pred

Basic algorithm Optimizations: Which exploration strategy to use? Random Top-bottom Bottom-top Midpoint What information to reuse? Losing states from below Winning states from above State space from below GASICS Workshop19 {φ}{φ} {φ} U Pred

Basic algorithm Optimizations: Which exploration strategy to use? Random Top-bottom Bottom-top Midpoint What information to reuse? Losing states from below Winning states from above State space from below GASICS Workshop20 {φ}{φ} {φ} U Pred

Basic algorithm Optimizations: Which exploration strategy to use? Random Top-bottom Bottom-top Midpoint What information to reuse? Losing states from below Winning states from above State space from below GASICS Workshop21 {φ}{φ} {φ} U Pred

Basic algorithm Optimizations: Which exploration strategy to use? Random Top-bottom Bottom-top Midpoint What information to reuse? Losing states from below Winning states from above State space from below GASICS Workshop22 {φ}{φ} {φ} U Pred

State space reusage GASICS Workshop23 {φ}{φ} {φ} U Pred a a a b a b L 1, x≥4 L 2, x≥5 L 3, x<2 L 4, x≥8 L 5, x≥7 L 6, x<2 (L 1, x≥4) ∨ (L 2, x≥5) ∨ (L 3, x<2) (L 4, x≥8) ∨ (L 5, x≥7) ∨ (L 6, x<2) L 6, x<2

State space reusage GASICS Workshop24 {φ}{φ} {φ} U Pred a a a b a b L 1, x≥4 L 2, x≥5 L 3, x<2 L 4, x≥8 L 5, x≥7 L 6, x<2 (L 1, x≥4) ∨ (L 2, x≥5) ∨ (L 3, x<2) (L 4, x≥8) ∨ (L 5, x≥7) ∨ (L 6, x<2) L 6, x<2

Implementation details 25 Efficient Stable Ready for industry applications Has a nice GUI Easy to prototype new very specific features

Python framework for timed automata manipulation PyDBM – Python wrapper for UPPAAL DBM library pyuppaal – syntactic parser of UPPAAL models dbmpyuppaal – parses a model using pyuppaal and replaces all guards and invariants by their DBMs opaal – model checker for timed automata More information at: GASICS Workshop26

Results GASICS Workshop27 EJECT RESET Possible observations and their cost: {H -> 1, L ->1, y≥1 -> 10, y≥2 -> 9, …, y≥10 -> 1} Optimal solution: {H, y≥5}

Results (average running time) GASICS Workshop28

Questions? GASICS Workshop29