1. Solving Games Without Determinization Nir Piterman École Polytechnique Fédéral de Lausanne (EPFL) Switzerland Joint work with Thomas A. Henzinger

2. Nondeterminizing Nondeterministic Automata Nir Piterman École Polytechnique Fédéral de Lausanne (EPFL) Switzerland Joint work with Thomas A. Henzinger

3. What? Get a nondeterministic automaton with n states. Construct a nondeterministic automaton with 2 n n 2n states. Why?

4. Plan of Talk Verification. Automata on Infinite Words. Synthesis. Design Synthesis in Action. Our solution.

5. Verification The normal process of development: –Write specifications (informally). –Develop design. –Test. Check that the system satisfies the specification.

6. Reactive Systems We are interested in systems that behave rather than compute (CPU, Operating system). Main complexity is in maintaining communication with a user / another program / the environment. The system has to be ready for every possible input. The system maintains behavior forever.

7. What is Behavior? The sequence of states the system passes along a computation. Nondeterministic systems / many possible inputs produce many possible behaviors. For reactive systems the behavior is infinite.

8. Automata Theoretic Approach to Verification Use automata to reason about systems and specifications. Questions like satisfiability and model checking reduce to emptiness of automata. Separates logical and algorithmic aspects of problems.

9. Automata on Infinite Words Introduced by Büchi, McNaughton, Elgot, Trakhtenbrot, Rabin, … in the 60s. Basically: take the same machine; run it on infinite words. In infinite runs there is no last state. Use the set of recurring states. Büchi acceptance: the set of recurring states intersects the set of accepting states.

10. Examples q0q0 q1q1

11. q0q0 q1q1

12. Applications Satisfiability of S1S [Buc62] and linear time logics. –A linear time formula characterizes sets of sequences. –Construct an automaton that accepts the set of models of the formula. –Is the language of the automaton empty?

13. Applications Linear-time model checking [VW94]. –A linear time formula characterizes sets of sequences. –Construct an automaton that accepts all non- models of the formula. –Consider the intersection of the automaton and the system. –Is the intersection empty?

14. Verification The normal process of development: –Write specifications (informally). –Develop design. –Test. Check that the system satisfies the specification. We need a formal way to write specifications: temporal logic.

15. Specifications We formally write specifications using temporal logic. We use automata on infinite words as an intermediate tool to reason about specifications.

16. Synthesis Can’t we automatically produce the system from the specification? Produce systems that are ensured to work correctly.

17. Church’s Problem In 1965 Church posed this problem as: Given a circuit interface and a behavioral specification, determine: 1.Does there exist an automaton (circuit) that realizes the specification? 2. Construct an implementing circuit.

18. Solutions Rabin develops the theory of automata on infinite trees [Rab69]. Büchi and Landweber propose a reduction to infinite duration games [BL69]. These are the main two solutions up till today.

19. Synthesis as a Game System controls internal variables. Environment controls input. Moves of system must match all possible future moves of environment. System plays against environment. –System tries to satisfy specification. –Environment tries to falsify specification. Success of system determined by the outcome of interaction.

20. Game Graphs We represent games as directed graphs. G= h V,V 0,V 1,E,v 0 i The vertices are partitioned to those of player 0 (system) and player 1 (environment). A play starts with a pebble on v 0. If the pebble is on v 2 V 0, player 0 chooses an outgoing edge and transfers the pebble. If the pebble is on v 2 V 1, player 1 chooses the successor.

22. Winning Condition An infinite play is an infinite sequence of states. Winning conditions: –Recurrence / persistence in terms of states of the game. –Linear temporal logic or automata on infinite words over states of the game. Does there exist a winning strategy? Use the automaton to follow the play and determine the winner?

23. Use Automaton Add one pebble on the automaton. Move the pebble on the automaton according to the move in the game. Decide acceptance according to the automaton. Environment System Game Automaton

24. Simple Game 1 01 Visit finitely many 0’s Environment System

25. Nondeterminism is bad 1 01 Environment System

26. What’s the Problem? The opponent chooses between (infinitely) many different paths. A guess should match all possible paths. Deterministic automata don’t guess!

27. Determinization Need stronger acceptance conditions [Lan69]. Starting with NBW with n states: –DRW with 2 2 n states [McN66]. –DRW with (12) n n 2n states and 2n index [Saf88]. –DPW with n 2n+2 states and 2n index [Pit06]. Lower bound n O(n) [Mic88,Yan06]

28. Back to Games Games: –The opponent chooses between many different paths. –A deterministic automaton enables monitoring the goal of the game. Games with LTL/NBW goals: –Convert LTL to NBW, convert NBW to DPW. –Create product of game and DPW. Reasoning about general games reduces to reasoning about parity games.

29. The End?! Not really …

30. In Practice Determinization is extremely complex.

31. Safra’s Construction Have a tree of subset constructions. Whenever a node (subset) visits F, create a new son with the states in F. If a node is removed – flash red light. If a node equals its sons – flash green light. The Rabin condition has a pair for every node. Node flashes red – bad. Node flashes green – good.

32. Deterministic State Ordered tree. Nodes are elements in {1,…,n}. Every node is labeled by a subset of the states. Every node is colored green, red, or white. Unused names are colored red.

33. Deterministic Transition The transition of d is the result of the following transformations. Replace node label by labels of successors (subset construction). Spawn new sons with accepting states. Move states to ‘best’ nodes. Remove empty nodes. Nodes that equal their sons colored green. 0,1,3 3 1 1 4 2 3 1 1 4 2 1 1 3 5 3 1 1 4 1 5 2 3 1 1 4 0,1,3,4 4 1 2 1 4

34. What about your variant? Recently, improvement of Safra: –Safra: NBW(n) ! DRW(12 n n 2n,n) –Variant: NBW(n) ! DPW(n 2n+2,2n) But: still trees, and everything else.

35. Or abcdefghij

36. In Practice Determinization is extremely complex. First implementation in CIAA05.

37. OmegaDet [STW05]

38. In Practice Determinization is extremely complex. First implementation in CIAA05. No way to implement symbolically. All or nothing. Resort to other solutions.

39. Restrict attention to a subset of LTL. –Safety / reachability – linear time [RW89,AMPS98]. –Recurrence / persistance – quadratic time [AMPS98]. –Boolean combinations of safety / reachability [AT04]. –Generalized Reactivity(1) – cubic time [PPS06]. Practical Solution 1

40. Practical Solution 2 [JGB05,HRS05] Heuristics that use the NBW. Works? Good. Does not work?

41. Nondeterminism Nondeterministic automata cannot be used for game monitoring. Or can they? They just have to be built correctly…

42. Good for Games Automata Automata that can be controlled in a step-wise fashion. Defined via a game on the structure of the automaton. Can be used for game monitoring. Environment System Game Automaton

43. Definition Define the monitor game played on the structure of the automaton: –Start from the initial state. –Opponent chooses a letter. –We choose successor. –We win if: The resulting word is not in the language The resulting run is accepting An automaton is GFG if we win from initial state.

44. 1111111 · · · · 11 0 1111111 · · · ·

45. 2 1 3 0,1 1 1 0

46. Use for Game Monitoring Given a GFG we combine the game with the GFG. Player 0 chooses how to advance the GFG. Environment System Game Automaton

47. Where do I get one? Prove that an automaton is good for games if it fair-simulates another good for games. Deterministic automata are trivially good for games. So start from the deterministic automaton. We show how to construct one.

48. Construct a GFG Automaton Replace the tree structure by nondeterminism. Follow nondeterministically n subsets of states. Ensure that all the runs followed by some subset visit accepting states infinitely often. Wrong guess? Change your mind! Intuition: - first set is the subset construction. - other n-1 sets follow subsets of first set.

49. Construct a GFG Let’s start with details on determinization.

50. Determinization in Detail 0,1 1 1 a b ab a 1 1 0 0 Subset Construction There are infinitely many runs that reach an accepting state a finite number of times. Somehow these runs have to be separated.

51. Determinization Construction Have a tree of subset constructions. Whenever a node (subset) visits F, create a new son with the states in F. If a node is removed – flash red light. If a node equals its sons – flash green light. The parity condition follows the minimal node that flashed red/green infinitely often.

53. What is a state A tree. Nodes are elements in {1,…,n}. Every node is labeled by a subset of the states. G 2 {1,...,n+1} - the least node colored green. R 2 {1,…,n+1} – the least node that got erased.

54. Transition Replace label by the set of successors (subset construction). Create youngest son with subset of accepting states. Move double states to older brothers. If node equal to union of sons, remove sons and color green. Remove empty nodes. Compact names.

55. 0,1,3,4 4 0,3 b 0,1,3,4 4 subset construction 0,1,3,4 4 4,1 spawn sons 4 0,1,3,4 4 move to older sons 4 1 0,1,3,4 4 Handle full nodes 1 remove empty nodes c 1 2 1 2 1 1 2 1 3 4 2 1 3 4 2 1 4 0,1,3 3 1 1 4 subset construction 2 0,1,3 3 1 1 4 2 spawn sons 1 1 3 5 0,1,3 3 1 1 4 1 5 move to older sons 2 0,1,3 3 1 1 4 Handle full nodes a subset construction 2 2

56. From OmegaDet [STW05] 1 0 1 0 1 1 0 0

57. Safra from a node’s point of view I follow some states. Some of them may disappear. If all visit acceptance set, I raise a green flag. If all disappear I die. After I die, I can be revived with a new set.

58. Our Construction A State Up to n subsets of the states of the NBW. Every state in a subset is either marked or unmarked. If a subset is empty all subsets above it are empty.

59. Our Construction A Transition Replace every set with a subset of the possible successors. Successors of marked states are marked; accepting states are marked. If all are marked, remove marking. An empty set can load a subset of the first set.

60. Advantages Very simple construction. Amenable to symbolic implementation. Natural incremental structure leading to complete solution.

61. A Range of Constructions We can get closer / further from the deterministic automaton. The number of states goes between n 2n and n 3n. It all depends on the symbolic implementation…

62. Incremental Construction We don’t always need n sets. An automaton with i+1 sets ‘monitors fully’ more games than an automaton with i sets. It depends on the game itself. It is not related (directly) to memory.

63. Summary Replace deterministic automata by nondeterministic automata. Definition of GFG automata. Construction of GFG automata. Simple, amenable to symbolic implementation. Incremental structure leading to the full solution. Initial enumerative implementation. Lower bound.

64. Safraless Decision Procedures [KV05] Emptiness of alternating parity tree automata by rank computation. Requires determinization for the upper bound. Reduces to Büchi games instead of parity. Complexity may be quadratically worse. Strategy may be exponentially worse. Enables solution of games with LTL winning conditions. Does not apply for NBW winning conditions. Does not apply to infinite structures.

65. Future Work Implementation. Reuse work done in increments. Understand better the incremental structure. Automata for the complement language. Lower bound on the index.

66. Going Both Ways It would be nice to find both winning and losing states fast. Starting from LTL it is easy. –Build NBW N  for . –Build NBW N :  for : . –Combine the game incrementally with GFG for N . –Combine the game incrementally with GFG for N : . Starting from NBW? –Build GFG for N. –Build KV ranks for N.

67. Thank You