Presentation is loading. Please wait.

Presentation is loading. Please wait.

Zadar, August 2010 1 Learning probabilistic finite automata Colin de la Higuera University of Nantes.

Published byGiles Fletcher Modified over 8 years ago

Similar presentations

Presentation on theme: "Zadar, August 2010 1 Learning probabilistic finite automata Colin de la Higuera University of Nantes."— Presentation transcript:

1 Zadar, August 2010 1 Learning probabilistic finite automata Colin de la Higuera University of Nantes

2 Zadar, August 20102 Acknowledgements Laurent Miclet, Jose Oncina, Tim Oates, Rafael Carrasco, Paco Casacuberta, Rémi Eyraud, Philippe Ezequel, Henning Fernau, Thierry Murgue, Franck Thollard, Enrique Vidal, Frédéric Tantini,... List is necessarily incomplete. Excuses to those that have been forgotten http://pagesperso.lina.univ-nantes.fr/~cdlh/slides/ Chapters 5 and 16

3 Zadar, August 20103 Outline 1. PFA 2. Distances between distributions 3. FFA 4. Basic elements for learning PFA 5. ALERGIA 6. MDI and DSAI 7. Open questions

4 Zadar, August 2010 4 1 PFA Probabilistic finite (state) automata

5 Zadar, August 20105 Practical motivations (Computational biology, speech recognition, web services, automatic translation, image processing …) A lot of positive data Not necessarily any negative data No ideal target Noise

6 Zadar, August 20106 The grammar induction problem, revisited The data consists of positive strings, «generated» following an unknown distribution The goal is now to find (learn) this distribution Or the grammar/automaton that is used to generate the strings

7 Zadar, August 20107 Success of the probabilistic models n-grams Hidden Markov Models Probabilistic grammars

8 Zadar, August 20108 a b a b a b DPFA: Deterministic Probabilistic Finite Automaton

9 Zadar, August 20109 a b a b a b Pr A (abab)=

10 Zadar, August 201010 0.1 0.3 a b a b a b 0.65 0.35 0.9 0.7 0.3 0.7

11 Zadar, August 201011 b b a a a b PFA: Probabilistic Finite (state) Automaton

12 Zadar, August 201012  b   a b  -PFA: Probabilistic Finite (state) Automaton with  -transitions

13 Zadar, August 201013 How useful are these automata? They can define a distribution over  * They do not tell us if a string belongs to a language They are good candidates for grammar induction There is (was?) not that much written theory

14 Zadar, August 201014 Basic references The HMM literature Azaria Paz 1973: Introduction to probabilistic automata Chapter 5 of my book Probabilistic Finite-State Machines, Vidal, Thollard, cdlh, Casacuberta & Carrasco Grammatical Inference papers

15 Zadar, August 201015 Automata, definitions Let D be a distribution over  * 0  Pr D (w)  1  w  * Pr D (w)=1

16 Zadar, August 201016 A Probabilistic Finite (state) Automaton is a Q set of states I P : Q  [0;1] F P : Q  [0;1]  P : Q  Q  [0;1]

17 Zadar, August 201017 What does a PFA do? It defines the probability of each string w as the sum (over all paths reading w) of the products of the probabilities Pr A (w)=   i  paths(w) Pr(  i )  i =q i 0 a i 1 q i 1 a i 2 …a i n q i n Pr(  i )=I P (q i 0 )·F P (q i n ) ·  ai j  P (q i j-1,a i j,q i j ) Note that if -transitions are allowed the sum may be infinite

18 Zadar, August 201018 Pr(aba) = 0.7*0.4*0.1*1 +0.7*0.4*0.45*0.2 = 0.028+0.0252=0.0532 0.2 0.1 1 a b a a a b 0.45 0.35 0.4 0.7 0.3 0.1 b 0.4

19 Zadar, August 201019 non deterministic PFA: many initial states/only one initial state an -PFA: a PFA with -transitions and perhaps many initial states DPFA: a deterministic PFA

20 Zadar, August 201020 Consistency A PFA is consistent if Pr A (  * )=1  x  * 0  Pr A (x)  1

21 Zadar, August 201021 Consistency theorem A is consistent if every state is useful (accessible and co- accessible) and  q  Q F P (q) +  q’  Q,a   P (q,a,q’)= 1

22 Zadar, August 201022 Equivalence between models Equivalence between PFA and HMM… But the HMM usually define distributions over each  n

23 Zadar, August 201023 win draw lose 3434 1212 1414 1414 3434 1414 1414 A football HMM

24 Zadar, August 201024 Equivalence between PFA with -transitions and PFA without -transitions cdlh 2003, Hanneforth & cdlh 2009 Many initial states can be transformed into one initial state with -transitions; -transitions can be removed in polynomial time; Strategy: number the states eliminate first -loops, then the transitions with highest ranking arrival state

25 Zadar, August 201025 PFA are strictly more powerful than DPFA Folk theorem (and) You can’t even tell in advance if you are in a good case or not (see: Denis & Esposito 2004)

26 Zadar, August 201026 a a a a This distribution cannot be modelled by a DPFA Example

27 Zadar, August 201027 What does a DPFA over  ={a} look like? And with this architecture you cannot generate the previous one a … a a a

28 Zadar, August 201028 Parsing issues Computation of the probability of a string or of a set of strings Deterministic case Simple: apply definitions Technically, rather sum up logs: this is easier, safer and cheaper

29 Zadar, August 201029 Pr(aba) = 0.7*0.9*0.35*0 = 0 Pr(abb) = 0.7*0.9*0.65*0.3 = 0.12285 0.1 0.3 a b a b a b 0.65 0.35 0.9 0.7 0.3 0.7

30 Zadar, August 201030 Pr(aba) = 0.7*0.4*0.1*1 +0.7*0.4*0.45*0.2 = 0.028+0.0252=0.0532 Non-deterministic case 0.2 0.1 1 a b a a a b 0.45 0.35 0.4 0.7 0.3 0.1 b 0.4

31 Zadar, August 201031 In the literature The computation of the probability of a string is by dynamic programming : O(n 2 m) 2 algorithms: Backward and Forward If we want the most probable derivation to define the probability of a string, then we can use the Viterbi algorithm

32 Zadar, August 201032 Forward algorithm A[i,j]=Pr(q i |a 1..a j ) (The probability of being in state q i after having read a 1..a j ) A[i,0]=I P (q i ) A[i,j+1]=  k  |Q| A[k,j].  P (q k,a j+1,q i ) Pr(a 1..a n )=  k  |Q| A[k,n]. F P (q k )

33 Zadar, August 2010 33 2 Distances What for? Estimate the quality of a language model Have an indicator of the convergence of learning algorithms Construct kernels

34 Zadar, August 201034 2.1 Entropy How many bits do we need to correct our model? Two distributions over  * : D et D’ Kullback Leibler divergence (or relative entropy) between D and D’ :  w  * Pr D (w)  log Pr D (w)-log Pr D’ (w) 

35 Zadar, August 201035 2.2 Perplexity The idea is to allow the computation of the divergence, but relatively to a test set (S) An approximation (sic) is perplexity: inverse of the geometric mean of the probabilities of the elements of the test set

36 Zadar, August 201036  w  S Pr D (w) -1/  S  = 1  w  S Pr D (w) Problem if some probability is null... SS

37 Zadar, August 201037 Why multiply (1) We are trying to compute the probability of independently drawing the different strings in set S

38 Zadar, August 201038 Why multiply? (2) Suppose we have two predictors for a coin toss Predictor 1: heads 60%, tails 40% Predictor 2: heads 100% The tests are H: 6, T: 4 Arithmetic mean P1: 36%+16%=0,52 P2: 0,6 Predictor 2 is the better predictor ;-)

39 Zadar, August 201039 2.3 Distance d 2 d 2 ( D, D’ )=  w  * (Pr D (w)-Pr D’ (w)) 2 Can be computed in polynomial time if D and D’ are given by PFA (Carrasco & cdlh 2002) This also means that equivalence of PFA is in P

40 Zadar, August 2010 40 3 FFA Frequency Finite (state) Automata

41 Zadar, August 201041 A learning sample is a multiset Strings appear with a frequency (or multiplicity) S={ (3), aaa (4), aaba (2), ababa (1), bb (3), bbaaa (1)}

42 Zadar, August 201042 DFFA A deterministic frequency finite automaton is a DFA with a frequency function returning a positive integer for every state and every transition, and for entering the initial state such that the sum of what enters is equal to what exits and the sum of what halts is equal to what starts

43 Zadar, August 201043 Example 3 1 2 b : 5 b: 3 a: 1a: 2 a: 5 b: 4 6

44 Zadar, August 201044 From a DFFA to a DPFA 3/13 1/6 2/7 b: 5/13 b: 3/6 a: 1/7a: 2/6 a: 5/13 b: 4/7 6/6 Frequencies become relative frequencies by dividing by sum of exiting frequencies

45 Zadar, August 201045 From a DFA and a sample to a DFFA 3 1 2 b: 5 b: 3 a: 1a: 2 a: 5 b: 4 6 S = {, aaaa, ab, babb, bbbb, bbbbaa}

46 Zadar, August 201046 Note Another sample may lead to the same DFFA Doing the same with a NFA is a much harder problem Typically what algorithm Baum-Welch (EM) has been invented for…

47 Zadar, August 201047 The frequency prefix tree acceptor The data is a multi-set The FTA is the smallest tree-like FFA consistent with the data Can be transformed into a PFA if needed

48 Zadar, August 201048 From the sample to the FTA 3 2 4 3 a:7 a:6 a:4 a:2 b:4 b:2 1 a:1 b:1 1 a:1 b:1 a:1 S={ (3), aaa (4), aaba (2), ababa (1), bb (3), bbaaa (1)} FTA(S) 14

49 Zadar, August 201049 Red, Blue and White states a a a b b b a b a -Red states are confirmed states -Blue states are the (non Red) successors of the Red states -White states are the others Same as with DFA and what RPNI does

50 Zadar, August 201050 Merge and fold 60 9 10 11 6 4 Suppose we decide to merge with state a:26 a:4 a:6 a:4 a:10 b:24 b:9 b:6 100 b a

51 Zadar, August 201051 Merge and fold 60 9 10 11 6 4 First disconnect and reconnect to a:26 a:4 a:6 a:4 a:10 b:24 b:9 b:6 100 b a

52 Zadar, August 201052 Merge and fold 60 9 10 11 6 4 Then fold a:26 a:4 a:6 a:4 a:10 b:24 b:9 b:6 100

53 Zadar, August 201053 Merge and fold 60 10 9 11 4 after folding a:26 a:4 a:10 b:24 b:9 b:30 100

54 Zadar, August 201054 State merging algorithm A=FTA(S); Blue ={  (q I,a): a  }; Red ={q I } While Blue  do choose q from Blue such that Freq(q)  t 0 if  p  Red: d(A p,A q ) is small then A = merge_and_fold(A,p,q) else Red = Red  {q} Blue = {  (q,a): q  Red} – {Red}

55 Zadar, August 201055 The real question How do we decide if d(A p,A q ) is small? Use a distance… Be able to compute this distance If possible update the computation easily Have properties related to this distance

56 Zadar, August 201056 Deciding if two distributions are similar If the two distributions are known, equality can be tested Distance (L 2 norm) between distributions can be exactly computed But what if the two distributions are unknown?

57 Zadar, August 201057 Taking decisions 60 9 10 11 6 4 Suppose we want to merge with state a:26 a:4 a:6 a:4 a:10 b:24 b:9 b:6 100 b a

58 Zadar, August 201058 Taking decisions 60 9 10 11 6 4 Yes if the two distributions induced are similar a:26 a:4 a:6 a:4 a:10 b:24 b:9 b:6 9 11 4 a:4 b:24 b:9

59 Zadar, August 2010 59 5 Alergia

60 Zadar, August 201060 Alergia’s test D 1  D 2 if  x Pr D1 (x)  Pr D2 (x) Easier to test: Pr D1 ( )=Pr D2 ( )  a  Pr D1 (a  *)=Pr D2 (a  *) And do this recursively! Of course, do it on frequencies

61 Zadar, August 201061 Hoeffding bounds  indicates if the relative frequencies and are sufficiently close

62 Zadar, August 201062 A run of Alergia Our learning multisample S={ (490), a(128), b(170), aa(31), ab(42), ba(38), bb(14), aaa(8), aab(10), aba(10), abb(4), baa(9), bab(4), bba(3), bbb(6), aaaa(2), aaab(2), aaba(3), aabb(2), abaa(2), abab(2), abba(2), abbb(1), baaa(2), baab(2), baba(1), babb(1), bbaa(1), bbab(1), bbba(1), aaaaa(1), aaaab(1), aaaba(1), aabaa(1), aabab(1), aabba(1), abbaa(1), abbab(1)}

63 Zadar, August 201063 Parameter  is arbitrarily set to 0.05. We choose 30 as a value for threshold t 0. Note that for the blue state who have a frequency less than the threshold, a special merging operation takes place

64 Zadar, August 201064 490 10 8 31 128 38 170 a :257 a : 57 a : 15 b : 170 b : 26 b : 18 9 a : 13 a :5 4 42 b : 65 a : 14 b : 9 14 a : 64 10 4 3 6 b : 6 b : 7 3 2 2 2 2 1 1 1 2 2 2 1 2 1 1 1 1 1 1 1 1 1 1 a : 4 a : 5 a : 2 a : 4 a : 2 a : 1 b : 1 b : 2 b : 1 b : 2 b : 3 a b b b a a a a 1000

65 Zadar, August 201065 Can we merge and a? Compare and a, a  * and aa  *, b  * and ab  * 490/1000 with 128/257, 257/1000 with 64/257, 253/1000 with 65/257,.... All tests return true

66 Zadar, August 201066 490 10 8 31 128 38 170 a: 257 a: 57 a: 15 b: 170 b: 26 b: 18 9 a: 13 a: 5 4 42 b: 65 a: 14 b: 9 14 a: 64 10 4 3 6 b: 6 b: 7 3 2 2 2 2 1 1 1 2 2 2 1 2 1 1 1 1 1 1 1 1 1 1 a : 4 a : 5 a : 2 a : 4 a : 2 a : 1 b : 1 b : 2 b : 1 b : 2 b : 3 a b b b a a a a Merge… 1000

67 Zadar, August 201067 660 52 225 a:341 a: 77 b: 340 b: 38 12 a: 16 a: 10 20 7 6 7 b: 9 b: 8 2 1 1 1 2 1 1 a: 2 a: 1 b: 1 b: 2 And fold 1000

68 Zadar, August 201068 660 52 225 a:341 a: 77 b: 340 b: 38 12 a: 16 a: 10 20 7 6 7 b: 9 b: 8 2 1 1 1 2 1 1 a : 2 a : 1 b : 1 b : 2 Next merge ? with b ? 1000

69 Zadar, August 201069 Can we merge and b? Compare and b, a  * and ba  *, b  * and bb  * 660/1341 and 225/340 are different (giving  = 0.162) On the other hand

70 Zadar, August 201070 660 52 225 a:341 a: 77 b: 340 b: 38 12 a: 16 a:10 20 7 6 7 b: 9 b: 8 2 1 1 1 2 1 1 a : 2 a : 1 b : 1 b : 2 Promotion 1000

71 Zadar, August 201071 660 52 225 a: 341a: 77 b: 340 b: 38 12 a: 16 a: 10 20 7 6 7 b: 9 b: 8 2 1 1 1 2 1 1 a : 2 a : 1 b : 1 b : 2 Merge

72 Zadar, August 201072 660 291 a:341a: 95 b: 340 b: 49 a: 11 29 7 8 b: 9 2 1 2 a : 2 a : 1 b : 2 And fold 1000

73 Zadar, August 201073 660 225 a:341a: 95 b: 340 b: 49 a: 11 29 7 8 b: 9 2 1 2 a : 2 a : 1 b : 2 Merge 1000

74 Zadar, August 201074 698302 a: 354a: 96 b: 351 b: 49 And fold 1000.698.302 a:.354a:.096 b:.351 b:.049 As a PFA

75 Zadar, August 201075 Conclusion and logic Alergia builds a DFFA in polynomial time Alergia can identify DPFA in the limit with probability 1 No good definition of Alergia’s properties

76 Zadar, August 2010 76 6 DSAI and MDI Why not change the criterion?

77 Zadar, August 201077 Criterion for DSAI Using a distinguishable string Use norm L  Two distributions are different if there is a string with a very different probability Such a string is called  -distinguishable Question becomes: Is there a string x such that |Pr A,q (x)-Pr A,q’ (x)|> 

78 Zadar, August 201078 (much more to DSAI) D. Ron, Y. Singer, and N. Tishby. On the learnability and usage of acyclic probabilistic finite automata. In Proceedings of Colt 1995, pages 31–40, 1995. PAC learnability results, in the case where targets are acyclic graphs

79 Zadar, August 201079 Criterion for MDI MDL inspired heuristic Criterion is: does the reduction of the size of the automaton compensate for the increase in preplexity? F. Thollard, P. Dupont, and C. de la Higuera. Probabilistic Dfa inference using Kullback-Leibler divergence and minimality. In Proceedings of the 17th International Conference on Machine Learning, pages 975–982. Morgan Kaufmann, San Francisco, CA, 2000

80 Zadar, August 2010 80 7 Conclusion and open questions

81 Zadar, August 201081 A good candidate to learn NFA is DEES Never has been a challenge, so state of the art is still unclear Lots of room for improvement towards probabilistic transducers and probabilistic context-free grammars

82 Zadar, August 2010 82 Appendix Stern Brocot trees Identification of probabilities If we were able to discover the structure, how do we identify the probabilities?

83 Zadar, August 201083 By estimation: the edge is used 1501 times out of 3000 passages through the state : 3000 a 1501 3000

84 Zadar, August 201084 Stern-Brocot trees: (Stern 1858, Brocot 1860) Can be constructed from two simple adjacent fractions by the «mean» operation aca+c bdb+d = m

85 Zadar, August 201085 0 1 1 0 1 12 21 1 2 33 3 3 21 12 334 5 5 4 45 543 3 2 1

86 Zadar, August 201086 Idea: Instead of returning c(x)/n, search the Stern-Brocot tree to find a good simple approximation of this value.

87 Zadar, August 201087 Iterated Logarithm: c(x) - a < log log n n b n  >1 With probability 1, for a co-finite number of values of n we have:

Download ppt "Zadar, August 2010 1 Learning probabilistic finite automata Colin de la Higuera University of Nantes."

Similar presentations

Recognising Languages We will tackle the problem of defining languages by considering how we could recognise them. Problem: Is there a method of recognising.

Recognising Languages We will tackle the problem of defining languages by considering how we could recognise them. Problem: Is there a method of recognising.
Characterization of state merging strategies which ensure identification in the limit from complete data Cristina Bibire.

Characterization of state merging strategies which ensure identification in the limit from complete data Cristina Bibire.
Lecture 24 MAS 714 Hartmut Klauck

Lecture 24 MAS 714 Hartmut Klauck
1 1 CDT314 FABER Formal Languages, Automata and Models of Computation Lecture 3 School of Innovation, Design and Engineering Mälardalen University 2012.

1 1 CDT314 FABER Formal Languages, Automata and Models of Computation Lecture 3 School of Innovation, Design and Engineering Mälardalen University 2012.
FORMAL LANGUAGES, AUTOMATA, AND COMPUTABILITY 15-453.

FORMAL LANGUAGES, AUTOMATA, AND COMPUTABILITY
1 Polynomial Time Probabilistic Learning of a Subclass of Linear Languages with Queries Yasuhiro TAJIMA, Yoshiyuki KOTANI Tokyo Univ. of Agri. & Tech.

1 Polynomial Time Probabilistic Learning of a Subclass of Linear Languages with Queries Yasuhiro TAJIMA, Yoshiyuki KOTANI Tokyo Univ. of Agri. & Tech.
Pushdown Automata Part II: PDAs and CFG Chapter 12.

Pushdown Automata Part II: PDAs and CFG Chapter 12.
Hidden Markov Model 11/28/07. Bayes Rule The posterior distribution Select k with the largest posterior distribution. Minimizes the average misclassification.

Hidden Markov Model 11/28/07. Bayes Rule The posterior distribution Select k with the largest posterior distribution. Minimizes the average misclassification.
1 Module 19 LNFA subset of LFSA –Theorem 4.1 on page 131 of Martin textbook –Compare with set closure proofs Main idea –A state in FSA represents a set.

1 Module 19 LNFA subset of LFSA –Theorem 4.1 on page 131 of Martin textbook –Compare with set closure proofs Main idea –A state in FSA represents a set.
61 Nondeterminism and Nodeterministic Automata. 62 The computational machine models that we learned in the class are deterministic in the sense that the.

61 Nondeterminism and Nodeterministic Automata. 62 The computational machine models that we learned in the class are deterministic in the sense that the.
Hidden Markov Models I Biology 162 Computational Genetics Todd Vision 14 Sep 2004.

Hidden Markov Models I Biology 162 Computational Genetics Todd Vision 14 Sep 2004.
Cohen, Chapter 61 Introduction to Computational Theory Chapter 6.

Cohen, Chapter 61 Introduction to Computational Theory Chapter 6.
COMMONWEALTH OF AUSTRALIA Copyright Regulations 1969 WARNING This material has been reproduced and communicated to you by or on behalf of Monash University.

COMMONWEALTH OF AUSTRALIA Copyright Regulations 1969 WARNING This material has been reproduced and communicated to you by or on behalf of Monash University.
Finite state automaton (FSA)

Finite state automaton (FSA)
1 Finite state automaton (FSA) LING 570 Fei Xia Week 2: 10/07/09 TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: AAA.

1 Finite state automaton (FSA) LING 570 Fei Xia Week 2: 10/07/09 TexPoint fonts used in EMF. Read the TexPoint manual before you delete this box.: AAA.
1 Hidden Markov Model Instructor : Saeed Shiry  CHAPTER 13 ETHEM ALPAYDIN © The MIT Press, 2004.

1 Hidden Markov Model Instructor : Saeed Shiry  CHAPTER 13 ETHEM ALPAYDIN © The MIT Press, 2004.
79 Regular Expression Regular expressions over an alphabet  are defined recursively as follows. (1) Ø, which denotes the empty set, is a regular expression.

79 Regular Expression Regular expressions over an alphabet  are defined recursively as follows. (1) Ø, which denotes the empty set, is a regular expression.
CS 454 Theory of Computation Sonoma State University, Fall 2011 Instructor: B. (Ravi) Ravikumar Office: 116 I Darwin Hall Original slides by Vahid and.

CS 454 Theory of Computation Sonoma State University, Fall 2011 Instructor: B. (Ravi) Ravikumar Office: 116 I Darwin Hall Original slides by Vahid and.
Lecture 5UofH - COSC 3340 - Dr. Verma 1 COSC 3340: Introduction to Theory of Computation University of Houston Dr. Verma Lecture 5.

Lecture 5UofH - COSC Dr. Verma 1 COSC 3340: Introduction to Theory of Computation University of Houston Dr. Verma Lecture 5.
Zadar, August 20101 Learning from an informant Colin de la Higuera University of Nantes.

Zadar, August Learning from an informant Colin de la Higuera University of Nantes.

Similar presentations

Ads by Google