Minimization of Symbolic Automata Presented By: Loris D’Antoni Joint work with: Margus Veanes 01/24/14, POPL14

What is automata minimization? 2

Deterministic Finite Automaton 3 a b q0q0 q a b A = (Q,q 0,F,δ,Σ)

Automata Minimization Minimization = find and collapse equivalent states 4 p q s s Non final Final distinguishable

a b a a,b b a b a b a b 01,32,46 5,6 a,b

A simple Application: Random Password generation Given constraints: Length is k: "^.{5,20}$" Contains 2 capital letters: "[A-Z].*[A-Z]" Contains a digit: "\d“ Generate random instances with uniform distribution that match all the above conditions. 6

Key idea ^.{5,20}$ [A-Z].*[A-Z] \d 7 ∩

Problems 8 Big automatonMinimization Big alphabet 2 16 characters in UTF16 Symbolic Automata

Symbolic Finite Automaton (SFA) 9 λx. x mod 2=0 λx. x mod 2=1 q0q0 q λx. x mod 2=0 λx. x mod 2=1 A = (Q,q 0,F,δ,σ) Input sort: in this case int Separate theory for the input alphabet SMT SOLVER

Symbolic Finite Automata (SFA) 10 λx. x mod 2=0 λx. x mod 2=1 p q λx. x mod 2 =0λx. x mod 2= ppqpp p is final  accept the input Execution Example

Advantages of Symbolic Automata Alphabet is represented symbolically – UTF16 abstracted using BDDs – Integer using predicates over integers Succinctness – at most n 2 transitions – One transition captures many symbols BUT: do DFA algorithms generalize to SFAs? 11

An example: SFA intersection 12 p1p1 q1q1 11 p2p2 q2q2 22 A1:A1: A2:A2: p1p2p1p2  1  2 A1A2:A1A2: q1q2q1q2 X delete when  1  2 unsatisfiable REQUIREMENTS: Input theory must be a Boolean algebra, and decidable

Moore’s algorithm 13 p q p’ q’ distinguishable a a n 2 iterations over k symbols O(kn 2 ) s s

Symbolic Moore’s algorithm Initially D = F x (Q\F) U (Q\F) x F for each (p’,q’) in D, (p,q) not in D let φ, ψ guards of δ (p,p’), δ (q,q’) if(isSat( φ ∧ ψ )) add (p,q) to D 14 p q p’ q’ distinguishable φ ψ φ ∧ ψ satisfiable m transitions O(m 2 f(k)) k = size of biggest predicate in SFA

Sometimes Moore is Less sec for 15 characters! the culprit should scale up to 128 characters!

Hopcroft’s algorithm: intuition 16 F Q\F

Hopcroft’s algorithm: intuition 17 a a a RA S

Hopcroft’s algorithm: intuition 18 P3P3 P2P2 P1P1 P4P4 R Keep partitioning with respect to W for every input symbol b b

Hopcroft’s algorithm: intuition 19 R Let’s assume I already split according to R P2P2 P1P1

Hopcroft’s algorithm: intuition 20 RQ Let’s assume I already split according to R P2P2 P1P1 Do I need to consider both P 1 and for P 2 future splitting?

Hopcroft’s algorithm: intuition 21 a a a RQ Let’s assume I already split according to R P2P2 P1P1 Do I need to consider both P 1 and for P 2 future splitting?

Hopcroft’s algorithm: intuition 22 a a a RQ Let’s assume I already split according to R P2P2 P1P1 Do I need to consider both P 1 and for P 2 future splitting?

Hopcroft’s algorithm: intuition 23 a a a RQ Let’s assume I already split according to R P2P2 P1P1 Do I need to consider both P 1 and for P 2 future splitting? NO I ONLY NEED ONE!

Hopcroft’s algorithm P := {F, Q\F} W := {if |F|< |Q\F| then F else Q\F} while W != { } R:=pickFrom(W) foreach a in Σ S := δ -1 (R,a) while ∃ T ∈ P. T ∩ S ≠ {} ∧ T \S ≠ {} P,W := split(P, P ∩ S, P\S) return partitioned DFA 24 log n iterations O(kn log n)

Hopcroft’s algorithm example a b a a,b P2P2 P1P1 b a b a b a b R PARTITION: {P 1, P 2 } TO ANALYZE: {P 2 }

Hopcroft’s algorithm example a b a a,b b a b a b a b R P2P2 P 11 P 12 PARTITION: {P 11, P 12, P 2 } TO ANALYZE: {P 2, P 12 }

Hopcroft’s algorithm example a b a a,b b a b a b a b R P2P2 P 11 P 12 PARTITION: {P 11, P 12, P 2 } TO ANALYZE: {P 12 }

Hopcroft’s algorithm example a b a a,b b a b a b a b 01,32,46 5,6 a,b

Symbolic Hopcroft’s algorithm P := {F, Q\F} W := {if |F|< |Q\F| then F else Q\F} while W != { } R:=pickFrom(W) foreach a in Σ S := δ -1 (R,a) while ∃ T ∈ P. T ∩ S ≠ {} ∧ T \S ≠ {} P,W := split(P, P ∩ S, P\S) return partitioned DFA 29 Alphabet might not be finite

Finitize the alphabet 30 φ1φ1 φ2φ2 φ3φ3 φ‘7φ‘7 φ'3φ'3 φ‘1φ‘1 φ‘4φ‘4 φ‘2φ‘2 φ‘5φ‘5 φ‘6φ‘6 φ‘8φ‘8 Predicates: {x>5, x<10, x=3} Minterms: {x=3, x≤5, 5<x<10, x≥10}

Symbolic Hopcroft’s algorithm P := {F, Q\F} W := {if |F|< |Q\F| then F else Q\F} while W ≠ {} R:=pickFrom(W) foreach φ in Minterms(A) S := δ -1 (R, φ ) while ∃ T ∈ P. T ∩ S ≠ {} ∧ T \S ≠ {} P,W := split(P, P ∩ S, P\S) return partitioned DFA 31 log n iterations O(2 m nlog n+2 m f(mk)) We need something better

New Algorithm: Intuition 32 Φ ψ A R P1P1 P2P2 p p q q What if Φ ≠ ψ? Φ\ψΦ\ψ

Example 1/ x<0 x≥0 -2<x<5 -5<x<3 -2<x<5 -5<x<3 true F Q\F false ≠ -5<x<3 R

Example 1/ x<0 x≥0 -2<x<5 -5<x<3 -2<x<5 -5<x<3 true R

Example 2/2 r 65 p q true x<2 x<5 x≥2 x≥5 Both p and q go to r, but… x≥2  x≥5 ?? NO Then p is distinguishable from q R

Example 2/2 r 65 p q true x<2 x<5 x≥2 x≥5 Both p and q go to r, but… x≥2  x≥5 ?? NO Then p is distinguishable from q R

New Algorithm P := {F, Q\F} W := {if |F|< |Q\F| then F else Q\F} while W ≠ { } R := pickFrom(W); S := δ -1 (R, true); while ∃ A ∈ P. A ∩ S ≠ {} ∧ ∃ p 1,p 2. δ -1 (p 1 ) ≠ δ -1 (p 2 ) P,W := split(P, P ∩ S, P\S, witness( δ -1 (p 1 ) ≠ δ -1 (p 2 ) ) return partitioned DFA 37 log n iterations O(n 2 log n f(nk))

Experiments 1.Randomly generated DFAs SFAs using BDDs (sort = bitvec 7 bits) 2.SFAs generated from regexes SFAs using BDDs (sort = bitvec 16 bits) 3.A corner case of Minterm generation SFAs using BDDs (sort = bitvec 20 bits) 4.Randomly generated SFAs over string x int SFAs over using Z3 (sort = string x int) 5.Monadic second order logic to DFA transformation SFAs using BDDs (sort = bitvec 40 bits)

1) Randomly generated DFAs 5 billion DFAs: 10 to 100 states, 2 to 50 symbols From [Almeida, Moreira, Reis, TR05]

2) SFAs generated from regexes (regexplib.com) 3000 regexes over UTF16 alphabet (2 16 elems) From [regexplib.com] Both axis logscale More States => Moore Worse

3) A corner case of Minterm generation This SFA has 2 k minterms!! brics.automata.dk Uses intervals instead of BDDs Logscale

4) Randomly generated SFAs over string x int Randomly generated 10 SFAs over string x int and minimized all the intersections, complement, difference, and union of such SFAs Random generation causes many predicate overlaps  minterms

5) MSO logic to DFA transformation [IJFCS05] State of the art for MSO

Conclusion Results Adapted classical minimization algorithm to the symbolic setting New minimization algorithm for symbolic automata (faster than previous ones) Future work Extend to tree automata Extend classical automata problems to SFAs – Edit distance? – Regex for symbolic automata? 44

Future Work Extending classical automata problems to SFAs – Edit distance? – Regex for symbolic automata? – Random generation of SFAs Using transducers (BEX) for inverting simple programs automatically. We already have some results on how to check injectivity QUESTIONS? 45

Application 1: Solving Monadic Second Order logic (MSO) MSO logic is equivalent to regular languages For example ∃ x,y. x<y ∧ a(x) ∧ b(y) Describes the DFA 0 1 a 2 b 2 b a a,b FO Variables = positions SO Variables = sets of positions

Monadic second order logic to DFA transformation Φ := Φ ∧ Φ | ¬ Φ | ∃ X.Φ | S(X) | X ⊆ Y | a(X) |X<Y For every subformula Φ we inductively compute the corresponding DFA, A(Φ). The first two are easy using automata operations (intersection complement) What about ∃ X.Φ ?

Monadic second order logic to DFA transformation For every formula Φ with free variables X 1,…,X n we extend the alphabet to model Φ Now we have a formula for Φ To compute ∃ X.Φ we remove the first element of the bitvector from every transition b0100b0100 Current position of the element being read: -is labeled with b, and -belongs to set X 2, but not X 1, X 3, X 4 X1X2X3X4X1X2X3X4

Monadic second order logic to DFA transformation The transformation is non-elementary n variables  n bits Secrets to make it work in practice Symbolic representation of alphabet, Minimize at every step We need a good representation of big alphabets, good minimization algorithms.