Program Synthesis is a Game Presented by: Sumit Nain Slides Credit: Barbara Jobstmann (CNRS/Verimag)

Outline Synthesis using automata-based game theory Lecture 2 Lecture 1 High Level Overview Motivate Model Checking, program repair/synthesis by analyzing example code Connection to games Not covered today: formal syntax, details of algorithms Lecture 2 Deeper dive into Automata-based game theory Formal terminology Reachability/safety, Buchi, parity games Algorithms for solving games

Analyze, Verify, and Synthesize Synthesis System Implementation Algorithm “How” Specification Properties Model “What” Verification Here is my picture of what the different techniques try to do Analysis Synthesis translates a high-level system description into an executable implementation Benefits: implementation and reward for writing specs (no double effort) Techniques vary depending on specification language and target implementation.

Reactive Systems Synthesis Verification Analysis Specifications Systems (e.g., servers) Interact with environment Infinite duration (non-terminating) Finite data (or data abstractions) Control-oriented Specifications Set of good behaviors (a language) Temporal logic (which includes safety and liveness) Verification How “good” does a system satisfies its spec Analysis [Z. Manna and A. Pnueli. The Temporal Logic of Reactive and Concurrent Systems., 1992] [N. Halbwachs. Synchronous Programming of Reactive Systems. 1993]

Verification Model Checking: Synthesis: Does system satisfy given specification Synthesis: Find a system that “satisfies” given specification

Example Specification Real life example from a specification for a high performance on-chip bus with multiple masters and an arbiter (ARM’s AMBA AHB bus) Specification: “When a locked unspecified length burst starts, a new access does not start until the current master (i) releases the bus by lowering HBUSREQi.”

Example: Formal Specification using Linear Temporal Logic (LTL) “When a locked unspecified length burst starts, a new access does not start until the current master (i) releases the bus by lowering HBUSREQi.” Formal specification: for all masters i: always((HMASTLOCK and HBURST=INCRand START and HMASTER=i) ➝ (not START until not HBUSREQi))‏ Time progresses with clock tick

Example: Software unsigned int got_lock = 0; ... while(*) { if (*) { lock(); got_lock++; } if (got_lock != 0) { unlock(); got_lock--; lock() {LOCK:=1;} unlock() {LOCK:=0;} Specifications P1: do not acquire a lock twice P2: do not call unlock without holding the lock reactive systems can also describe aspects of a concurrent software

Example: Software unsigned int got_lock = 0; ... 1: while(*) { 2: if (*) { 3: lock(); 4: got_lock++; } 5: if (got_lock != 0) { 6: unlock(); 7: got_lock--; lock() lock: {LOCK:=1;} unlock() unlock: {LOCK:=0;} Specifications P1: do not acquire a lock twice P2: do not call unlock without holding the lock P1: always( line=lock implies next( line!=lock wait-until line=unlock )) P2: ( line!=unlock wait-until line=lock )) and always( line=unlock implies next( line!=unlock wait-until line=lock )) Time progresses with change of line

How to verify a Reactive System? Core algorithm for Model Checking linear-time temporal logic: Source code modeled as transition system M (an FSM) Specification modeled as FSM A that accepts violations of the spec. 3. Check if model has a violating trace product of trans. system M and monitor A check for existence of a trace in product (check emptiness of M x A )

Source Code unsigned int got_lock = 0; ... 1: while(*) { 2: if (*) { 3: lock(); 4: got_lock++; } 5: if (got_lock != 0) { 6: unlock(); 7: got_lock--; lock() lock: {LOCK:=1;} unlock() unlock: {LOCK:=0;}

Step 1: Transition System Trans. system variables: line (l), got_lock (gl) int[0,1,2] got_lock = 0; ... 1: while(*) { 2: if (*) { 3: lock(); lock: {LOCK:=1;} 4: got_lock++; } 5: if (got_lock != 0) { 6: unlock(); unlock: {LOCK:=0;} 7: got_lock--; 8: l=1, gl=0 l=1, gl=1 l=1, gl=2 l=2, gl=0 .... l=2, gl=2 l=3, gl=0 l=3, gl=2 l=lock,gl=0 l=lock,gl=2 l=4, gl=0 l=4, gl=2 l=5, gl=0 l=5, gl=1 l=5, gl=2 l=6, gl=1 l=6, gl=2 l=unlock,gl=1 l=unlock,gl=2 l=7, gl=0 l=7, gl=1 l=7, gl=2 l=8, gl=0 .... ....

Specification P1: always( line=lock implies P1: do not acquire a lock twice P2: do not call unlock without holding the lock P1: always( line=lock implies next(line!=lock wait-until line=unlock)) P2: ( line!=unlock wait-until line=lock )) and always( line=unlock implies next( line!=unlock wait-until line=lock ))

Linear-Time Temporal Logic [Pnueli77] Syntax: Atomic propositions, e.g., line=1, line!=1, got_lock=0 Boolean operators: not, and, or, implies,... Temporal operators: next (ϕ) ... ϕ holds in the next step ϕ1 until ϕ2 ... ϕ1 holds until at some point ϕ2 holds Used in industrial spec languages PSL/SVA Can express many interesting properties, e.g., mutual exclusion, deadlock freedom, termination

Linear-Time Temporal Logic Semantics defined with respect to infinite traces in each step atomic propositions holds or not E.g., line=1, got_lock≤1 line=1 line=1 got_lock≤1 .....

Linear-Time Temporal Logic Semantics next (ϕ) ... ϕ holds in the next step ϕ1 until ϕ2 ... ϕ1 holds until at some point ϕ2 holds System S satisfies/models ϕ, if all its behaviors satisfy ϕ ϕ ..... ϕ1 ϕ1 ϕ1 ϕ1 ϕ2 ..... .....

Linear-Time Temporal Logic Many useful abbreviations true = p or not p false = not true eventually(ϕ) = true until ϕ always(ϕ) = not (eventually(not ϕ)) ϕ1 wait-until ϕ2 = (ϕ1 until ϕ2) or always(ϕ1)

How to verify a Reactive System? Core algorithm for linear-time temporal logic: Source code ➝auto/manual transition system M Specification ➝auto/manual monitor violations A Check if model has a violating trace Compute product of M and A Check product for emptiness

Step 2: Monitor for Violations P1: always( line=lock implies next( line!=lock wait-until line=unlock )) = not eventually( line=lock and next( line!=unlock until line=lock )) line!=unlock line=lock line=lock 1 2 3 s1: non-deterministic choice s2: no edge with line=unlock Automaton accepts trace/behavior is a blue state is visited infinitely often (Büchi )

Step 3: Product l=1, gl=0 l=1, gl=1 l=1, gl=2 l=2, gl=0 .... l=2, gl=2 l=lock,gl=0 l=lock,gl=2 l=4, gl=0 l=4, gl=2 l=5, gl=0 l=5, gl=1 l=5, gl=2 l=6, gl=1 l=6, gl=2 l=unlock,gl=1 l=unlock,gl=2 l=7, gl=0 l=7, gl=1 l=7, gl=2 line!=unlock line=lock line=lock 1 2 3 l=8, gl=0 .... ....

Step 3: Product l=1, gl=0 l=1, gl=1 l=1, gl=2 l=2, gl=0 .... l=2, gl=2 l=lock,gl=0 1 l=lock,gl=2 1 l=4, gl=0 l=4, gl=0 2 1 l=4, gl=2 1 l=5, gl=0 1 l=5, gl=1 l=5, gl=1 2 1 l=5, gl=2 1 l=6, gl=1 l=6, gl=1 2 1 l=6, gl=2 1 unlock,1 unlock,1 2 1 l=unlock,gl=2 1 l=7, gl=0 1 l=7, gl=1 1 l=7, gl=2 line!=unlock line=lock line=lock 1 2 3 1 l=8, gl=0 1 .... 1 ....

Recall, we want to show a violation: Step 3: Product 1 l=1, gl=0 1 l=1, gl=1 1 l=1, gl=2 l=1, gl=2 2 1 l=2, gl=0 1 .... 1 l=2, gl=2 l=2, gl=2 2 1 l=3, gl=0 1 l=3, gl=2 l=3, gl=2 2 1 l=lock,gl=0 1 l=lock,gl=2 l=lock,gl=2 2 1 l=4, gl=0 l=4, gl=0 2 1 l=4, gl=2 l=4, gl=2 2 l=4, gl=2 3 1 l=5, gl=0 l=5, gl=0 2 1 l=5, gl=1 l=5, gl=1 2 1 l=5, gl=2 1 l=6, gl=1 l=6, gl=1 2 1 l=6, gl=2 1 unlock,1 unlock,1 2 1 l=unlock,gl=2 1 l=7, gl=0 l=7, gl=0 2 1 l=7, gl=1 1 l=7, gl=2 line!=unlock line=lock line=lock 1 2 3 1 l=8, gl=0 1 .... 1 .... Recall, we want to show a violation:

Step 3: Product 1 l=1, gl=0 1 l=1, gl=1 1 l=1, gl=2 l=1, gl=2 2 1 l=2, gl=0 1 .... 1 l=2, gl=2 l=2, gl=2 2 1 l=3, gl=0 1 l=3, gl=2 l=3, gl=2 2 1 l=lock,gl=0 1 l=lock,gl=2 l=lock,gl=2 2 1 l=4, gl=0 l=4, gl=0 2 1 l=4, gl=2 l=4, gl=2 2 l=4, gl=2 3 1 l=5, gl=0 l=5, gl=0 2 1 l=5, gl=1 l=5, gl=1 2 1 l=5, gl=2 1 l=6, gl=1 l=6, gl=1 2 1 l=6, gl=2 1 unlock,1 unlock,1 2 1 l=unlock,gl=2 1 l=7, gl=0 l=7, gl=0 2 1 l=7, gl=1 1 l=7, gl=2 line!=unlock line=lock line=lock 1 2 3 1 l=8, gl=0 1 .... 1 .... Recall, we want to show a violation: non-determinism in transition system and in monitor pull in the same direction (both can be used to violate property)

Source Code line = 1 line = 2 line = 5 line = 7 line = 3 line = lock int[0,1,2] got_lock = 0; ... 1: while(*) { 2: if (*) { 3: lock(); lock: {LOCK:=1;} 4: got_lock++; } 5: if (got_lock != 0) { 6: unlock(); unlock: {LOCK:=0;} 7: got_lock--; 8:

How to verify a Reactive System? Core algorithm for linear-time temporal logic: Source code ➝auto/manual transition system M Specification ➝auto/manual monitor violations A Check if model has a violating trace Compute product of M and A Check product for emptiness But how to repair it?

How to repair a Reactive System? Add freedom (choice for the system, allowed ways to modify system) Source code ➝a/m transition system (game) Specification ➝a/m monitor acceptance Check if we can find system choices s.t. model is accepted by monitor product of trans. system and monitor search for winning strategy in game

Step 1: Freedom (We can also extend to fault localization) int[0,1,2] got_lock = 0; int[0,1,2] freedom; ... 1: while(*) { 2: if (*) { 3: lock(); lock: {LOCK:=1;} 4: got_lock:=freedom; } 5: if (got_lock != 0) { 6: unlock(); unlock: {LOCK:=0;} 7: got_lock:=freedom; 8: (We can also extend to fault localization)

Step 2: Game Trans. system variables: line (l), got_lock (gl) int[0,1,2] got_lock = 0; int[0,1,2] freedom; ... 1: while(*) { 2: if (*) { 3: lock(); lock: {LOCK:=1;} 4: got_lock:=freedom; } 5: if (got_lock != 0) { 6: unlock(); unlock: {LOCK:=0;} 7: got_lock:=freedom; 8: Trans. system variables: line (l), got_lock (gl) l=1, gl=0 l=1, gl=1 l=1, gl=2 l=2, gl=0 .... l=2, gl=2 l=3, gl=0 l=3, gl=2 l=lock,gl=0 l=lock,gl=2 f=0 f=2 f=1 l=4, gl=0 l=4, gl=2 l=5, gl=0 l=5, gl=1 l=5, gl=2 l=6, gl=1 l=6, gl=2 l=unlock,gl=1 l=unlock,gl=2 l=7, gl=0 l=7, gl=1 l=7, gl=2 l=8, gl=0 .... ....

Step 2: Game Trans. system variables: line (l), got_lock (gl) int[0,1,2] got_lock = 0; int[0,1,2] freedom; ... 1: while(*) { 2: if (*) { 3: lock(); lock: {LOCK:=1;} 4: got_lock:=freedom; } 5: if (got_lock != 0) { 6: unlock(); unlock: {LOCK:=0;} 7: got_lock:=freedom; 8: Trans. system variables: line (l), got_lock (gl) l=1, gl=0 l=1, gl=1 l=1, gl=2 l=2, gl=0 .... l=2, gl=2 l=3, gl=0 l=3, gl=2 l=lock,gl=0 l=lock,gl=2 f=0 f=2 f=1 l=4, gl=0 l=4, gl=2 l=5, gl=0 l=5, gl=1 l=5, gl=2 l=6, gl=1 l=6, gl=2 l=unlock,gl=1 l=unlock,gl=2 l=7, gl=0 l=7, gl=1 l=7, gl=2 l=8, gl=0 .... .... Two types of non-determinism!

Step 2: Game Trans. system variables: line (l), got_lock (gl) int[0,1,2] got_lock = 0; int[0,1,2] freedom; ... 1: while(*) { 2: if (*) { 3: lock(); lock: {LOCK:=1;} 4: got_lock:=freedom; } 5: if (got_lock != 0) { 6: unlock(); unlock: {LOCK:=0;} 7: got_lock:=freedom; 8: Trans. system variables: line (l), got_lock (gl) l=1, gl=0 l=1, gl=1 l=1, gl=2 l=2, gl=0 .... l=2, gl=2 l=3, gl=0 l=lock, gl=0 l=3, gl=2 l=lock, gl=2 f=0 f=2 f=1 l=4, gl=0 l=4, gl=2 l=5, gl=0 l=5, gl=1 l=6, gl=1 l=unlock, gl=1 l=5, gl=2 l=6, gl=2 l=unlock, gl=2 l=7, gl=0 l=7, gl=1 l=7, gl=2 l=8, gl=0 .... .... Two types of non-determinism!

Step 3: Monitor for Acceptance P1: always( line=lock implies next( line!=lock wait-until line=unlock )) line!=lock & line!=unlock line!=lock line=lock line=unlock line=lock

Step 3: Product TS for got_lock in {0,1} Deterministic automaton l=1, gl=0 l=1, gl=1 l=2, gl=0 l=1, gl=2 l=3, gl=0 l=lock, gl=0 l=3, gl=0 l=lock, gl=0 l!=lock & l!=unlock l!=lock l=lock l=4, gl=0 l=4, gl=0 1 2 3 1 1 l=5, gl=0 l=5, gl=1 l=6, gl=1 l=unlock, gl=1 l=unlock l=lock 1 1 l=7, gl=0 l=7, gl=1 l=8, gl=0 ....

Step 3: Product l=1, gl=0 l=1, gl=1 l=1, gl=0 l=1, gl=1 l=2, gl=0 l=lock, gl=0 1 l=3, gl=0 l=lock, gl=0 1 l=3, gl=0 l=lock, gl=0 2 l=3, gl=0 l=lock, gl=0 2 l=4, gl=0 1 l=4, gl=1 1 l=4, gl=0 2 l=4, gl=1 2 l=4, gl=.. 3 1 1 l=5, gl=0 1 l=5, gl=1 l=6, gl=1 l=unlock, gl=1 1 l=5, gl=0 2 l=5, gl=1 l=6, gl=1 l=unlock, gl=1 2 1 l=7, gl=0 1 l=7, gl=1 1 l=7, gl=0 2 l=7, gl=1 2 l=8, gl=0 1 l=8, gl=1 1 l=8, gl=0 2 l=8, gl=1 2 l!=lock & l!=unlock l!=lock l=lock 1 2 3 l=lock l=unlock

Step 4: Winning States l=1, gl=0 l=1, gl=1 l=1, gl=0 l=1, gl=1 2 l=1, gl=1 2 l=2, gl=0 1 l=2, gl=1 1 l=2, gl=0 2 l=2, gl=1 2 l=3, gl=0 l=lock, gl=0 1 l=3, gl=0 l=lock, gl=0 1 l=3, gl=0 l=lock, gl=0 2 l=3, gl=0 l=lock, gl=0 2 l=4, gl=0 1 l=4, gl=1 1 l=4, gl=0 2 l=4, gl=1 2 l=4, gl=.. 3 1 1 l=5, gl=0 1 l=5, gl=1 l=6, gl=1 l=unlock, gl=1 1 l=5, gl=0 2 l=5, gl=1 l=6, gl=1 l=unlock, gl=1 2 1 l=7, gl=0 1 l=7, gl=1 1 l=7, gl=0 2 l=7, gl=1 2 l=8, gl=0 1 l=8, gl=1 1 l=8, gl=0 2 l=8, gl=1 2 l!=lock & l!=unlock l!=lock l=lock 1 2 3 l=lock l=unlock

Step 4: Winning States l=1, gl=0 l=1, gl=1 l=1, gl=0 l=1, gl=1 2 l=1, gl=1 2 l=2, gl=0 1 l=2, gl=1 1 l=2, gl=0 2 l=2, gl=1 2 l=3, gl=0 l=lock, gl=0 1 l=3, gl=0 l=lock, gl=0 1 l=3, gl=0 l=lock, gl=0 2 l=3, gl=0 l=lock, gl=0 2 l=4, gl=0 1 l=4, gl=1 1 l=4, gl=0 2 l=4, gl=1 2 l=4, gl=.. 3 1 1 l=5, gl=0 1 l=5, gl=1 l=6, gl=1 l=unlock, gl=1 1 l=5, gl=0 2 l=5, gl=1 l=6, gl=1 l=unlock, gl=1 2 1 l=7, gl=0 1 l=7, gl=1 1 l=7, gl=0 2 l=7, gl=1 2 l=8, gl=0 1 l=8, gl=1 1 l=8, gl=0 2 l=8, gl=1 2 l!=lock & l!=unlock l!=lock l=lock 1 2 3 l=lock l=unlock

Step 4: Winning States l=1, gl=0 l=1, gl=1 l=1, gl=0 l=1, gl=1 2 l=1, gl=1 2 l=2, gl=0 1 l=2, gl=1 1 l=2, gl=0 2 l=2, gl=1 2 l=3, gl=0 l=lock, gl=0 1 l=3, gl=0 l=lock, gl=0 1 l=3, gl=0 l=lock, gl=0 2 l=3, gl=0 l=lock, gl=0 2 l=4, gl=0 1 l=4, gl=1 1 l=4, gl=0 2 l=4, gl=1 2 l=4, gl=.. 3 1 1 l=5, gl=0 1 l=5, gl=1 l=6, gl=1 l=unlock, gl=1 1 l=5, gl=0 2 l=5, gl=1 l=6, gl=1 l=unlock, gl=1 2 1 l=7, gl=0 1 l=7, gl=1 1 l=7, gl=0 2 l=7, gl=1 2 l=8, gl=0 1 l=8, gl=1 1 l=8, gl=0 2 l=8, gl=1 2 l!=lock & l!=unlock l!=lock l=lock 1 2 3 l=lock l=unlock

Step 4: Winning States l=1, gl=0 l=1, gl=1 l=2, gl=0 l=2, gl=1 l=lock, gl=0 1 l=3, gl=0 l=lock, gl=0 1 l=4, gl=0 1 l=4, gl=1 1 l=4, gl=0 2 l=4, gl=1 2 1 1 1 l=5, gl=0 1 l=5, gl=1 l=6, gl=1 l=unlock, gl=1 1 l=5, gl=1 l=6, gl=1 l=unlock, gl=1 2 1 l=7, gl=0 1 l=7, gl=1 1 l=8, gl=0 1 l=8, gl=1 1

Step 4: Winning Strategy In general: strategy is function of program and monitor state l=1, gl=0 1 l=1, gl=1 1 l=2, gl=0 1 l=2, gl=1 1 Strategy to Repair: if (l=4 & gl=0 & s=1) freedom:=0 if (l=4 & gl=1 & s=1) freedom:=1 if (l=4 & gl=0 & s=0) freedom:=1 if (l=7 & gl=0 & s=1) freedom:=0 if (l=7 & gl=1 & s=1) freedom:=0 .. freedom := f(l,gl,s) if (line=4) freedom := (gl=1) | (s=2) if (line=7) freedom := 0 l=3, gl=0 l=lock, gl=0 1 l=3, gl=0 l=lock, gl=0 1 l=4, gl=0 1 l=4, gl=1 1 l=4, gl=0 2 l=4, gl=1 2 1 1 1 l=5, gl=0 1 l=5, gl=1 l=6, gl=1 l=unlock, gl=1 1 l=5, gl=1 l=6, gl=1 l=unlock, gl=1 2 l=7, gl=0 1 l=7, gl=1 1 l=8, gl=0 1 l=8, gl=1 1 What we actually do: merge states before picking the strategy

Step 4: Winning Strategy l=1, gl=0 l=2, gl=0 l=3, gl=0 l=lock, gl=0 l=4, gl=0 l=5, gl=1 l=6, gl=1 l=unlock, gl=1 l=7, gl=1 l=8, gl=0 l=5, gl=0 l=7, gl=0 l=1, gl=1 l=2, gl=1 l=8, gl=1 l=4, gl=1 1 l=1, gl=0 1 l=1, gl=1 1 l=2, gl=0 1 l=2, gl=1 1 l=3, gl=0 l=lock, gl=0 1 l=3, gl=0 l=lock, gl=0 1 l=4, gl=0 1 l=4, gl=1 1 l=4, gl=0 2 l=4, gl=1 2 1 1 1 l=5, gl=0 1 l=5, gl=1 l=6, gl=1 l=unlock, gl=1 1 l=5, gl=1 l=6, gl=1 l=unlock, gl=1 2 1 l=7, gl=0 1 l=7, gl=1 1 l=8, gl=0 1 l=8, gl=1 1

Step 4: Winning Strategy l=1, gl=0 l=1, gl=1 l=2, gl=0 l=2, gl=1 l=3, gl=0 l=lock, gl=0 l=3, gl=0 l=lock, gl=0 (line=4): freedom= 1 (line=7): freedom= 0 l=4, gl=0 l=4, gl=1 1 1 l=5, gl=0 l=5, gl=1 l=6, gl=1 l=unlock, gl=1 l=7, gl=0 l=7, gl=1 l=8, gl=0 l=8, gl=1

Repaired Program unsigned int got_lock = 0; ... 1: while(*) { 2: if (*) { 3: lock(); 4: got_lock = 1; } 5: if (got_lock != 0) { 6: unlock(); 7: got_lock = 0; lock() lock: {LOCK = 1;} unlock() unlock: {LOCK = 0;}

Original Program unsigned int got_lock = 0; ... 1: while(*) { 2: if (*) { 3: lock(); 4: got_lock++; } 5: if (got_lock != 0) { 6: unlock(); 7: got_lock--; lock() lock: {LOCK = 1;} unlock() unlock: {LOCK = 0;}

Program Repair Program Finite-state program Game TS: program with freedom Solve game Monitor Monitor TS: Winning condition (Simple) Strategy Specification Correct Program with Bloem, Griesmayer, Staber in CAV 2005, CHARME 2005 (+ext to fault localization)

Synthesis from Temporal Logics Game Solve game Monitor + interface definition Monitor TS: Winning condition (Simple) Strategy Specification Correct Program Church (1962), Büchi/Landweber (1969, games), Rabin (1972, trees), Pnueli/Rosner (1989, LTL)

Program Synthesis Program FSM + freedom Game TS Solve game Monitor Monitor TS: Winning condition (Simple) Strategy Specification Correct Program Modern Controller Synthesis, see overview papers by Walukiewicz et al., Rutten & Girault, ...

Issues? Program FSM + freedom Game TS Solve game Monitor TS: Winning How to abstract? Game Size? How to solve efficiently? FSM + freedom Game TS Solve game How to construct efficiently? How expressive?Size? How to pick a strategy? Monitor Monitor TS: Winning condition (Simple) Strategy Quantitative Program Synthesis Specification How to map back? Correct Program How to specify? Related research areas: PL, AV, Control Theory, Game and Automata Theory LTL

Next Lecture Basic of automata-based game theory terminology reachability/safety, Buchi, parity games Algorithms for solving games