LING 388: Language and Computers Sandiway Fong Lecture 21: 11/7

2 Administrivia homework 6 –grammar programming –due Thursday lab class Thursday –Social Sciences 224

3 Review exploring questions of formal equivalence Regular Grammars FSARegular Expressions NDFSA FSA with -transitions x --> y, [t]. x --> [t]. (left recursive) or x --> [t], y. x --> [t]. (right recursive)

4 Review exploring questions of formal equivalence Regular Grammars (ND)FSA equivalent (right-recursive) regular grammar s --> [b], x. x --> [a], y. y --> [!]. y --> [a], y. sx z b ! y a a > Prolog FSA s([b|L]) :- x(L). x([a|L]) :- y(L). y([a|L]) :- y(L). y([‘!’|L]) :- z(L). z([]). queries ?- s(L). ?- s(L,[]). equivalent all the way down to when they output answers and loop

5 Review exploring questions of formal equivalence Regular Grammars s bx a y a y ! s !x ay a y b right recursive regular grammar s(s(b,X)) --> [b], x(X). x(x(a,Y)) --> [a], y(Y). y(y(!)) --> [!]. y(y(a,Y)) --> [a], y(Y). left recursive regular grammar s(s(X,!)) --> x(X),[!]. x(x(Y,a)) --> y(Y),[a]. y(y(b)) --> [b]. y(y(Y,a)) --> y(Y),[a]. same query ?- s(P,[b,a,a,!],[]). X = s(b,x(a,y(a,y(!)))) X = s(x(y(y(b),a),a),!) then semicolon ;( more answers ) terminates infinite loop

6 Last Time exploring questions of formal equivalence Regular Grammars the determiner-noun system –the man/men –a man/*a men DCG np(np(Y)) --> pronoun(Y). np(np(D,N)) --> det(D,Number), common_noun(N,Number). det(det(the),sg) --> [the]. det(det(the),pl) --> [the]. det(det(a),sg) --> [a]. common_noun(n(ball),sg) --> [ball]. common_noun(n(man),sg) --> [man]. common_noun(n(men),pl) --> [men]. pronoun(i) --> [i]. pronoun(we) --> [we]. DCG regular grammar equivalent np(np(i)) --> [i]. np(np(we)) --> [we]. np(np(the,N)) --> [the], common_nounsg(N). np(np(the,N)) --> [the], common_nounpl(N). np(np(a,N)) --> [a], common_nounsg(N). common_nounsg(n(ball)) --> [ball]. common_nounsg(n(man)) --> [man]. common_nounpl(n(men)) --> [men].

7 Determiner-Noun Agreement by extension –this determiner-noun system also has a FSA encoding determiner-noun regular grammar np(np(i)) --> [i]. np(np(we)) --> [we]. np(np(the,N)) --> [the], common_nounsg(N). np(np(the,N)) --> [the], common_nounpl(N). np(np(a,N)) --> [a], common_nounsg(N). common_nounsg(n(ball)) --> [ball]. common_nounsg(n(man)) --> [man]. common_nounpl(n(men)) --> [men]. np i common_nounsg > we the a ball man common_nounpl the men note: machine is a NDFSA. can be transformed into a deterministic machine

8 Determiner-Noun Agreement NDFSA ➟ FSA –set-of-states construction determiner-noun regular grammar np(np(i)) --> [i]. np(np(we)) --> [we]. np(np(the,N)) --> [the], common_nounsg(N). np(np(the,N)) --> [the], common_nounpl(N). np(np(a,N)) --> [a], common_nounsg(N). common_nounsg(n(ball)) --> [ball]. common_nounsg(n(man)) --> [man]. common_nounpl(n(men)) --> [men]. np i common_nounsg > we the a ball man common_nounpl the men np i common_nounsg > we a ball man {common_nounpl, common_nounsg} the men man ball

9 Determiner-Noun Agreement equivalence instantiated Regular Grammars FSARegular ExpressionsNDFSA FSA with -transitions np i common_nounsg > we a ball man {common_nounpl, common_nounsg} the men man ball np i common_nounsg > we the a ball man common_nounpl the men np(np(i)) --> [i]. np(np(we)) --> [we]. np(np(the,N)) --> [the], common_nounsg(N). np(np(the,N)) --> [the], common_nounpl(N). np(np(a,N)) --> [a], common_nounsg(N). common_nounsg(n(ball)) --> [ball]. common_nounsg(n(man)) --> [man]. common_nounpl(n(men)) --> [men]. we|i|(a (ball|man))|(the (ball|man|men))

10 Limits of Finite State Technology In general, it’s not true that grammars can always be rewritten into regular grammar form (and therefore simulatable by FSA)

11 Limits of Finite State Technology language = set of strings case 1 –suppose set is finite –e.g. L = {ba, abc, ccb, dd} easy to encode as a FSA (by closure under union) case 2 –set is infinite –... s1s1 s2s2 s3s3 ab s1s1 s2s2 s3s3 ba s4s4 c s1s1 s2s2 s3s3 cc s4s4 b s1s1 s2s2 s3s3 dd s0s0 ε ε ε ε

12 Limits of Finite State Technology Language = set of strings case 2 –set is infinite –e.g. L = a + b + – = { ab, aab, abb, aabb, aaab, abbb, … } “ one or more a ’ s followed by one or more b ’ s ” we know this set is regular –however, consider –L = {a n b n | n ≥ 1} – = { ab, aabb, aaabbb, aaaabbbb, … } “ the same number of b ’ s as a ’ s …” this set is not regular. Why? answer lies in the nature of the recursion sx y a a b b

13 The Limits of Finite State Technology [Aside: we can use the Pumping Lemma to prove this particular case.] informally, –we can build FSA for … –ab –aabb –aaabbb –… ab aabb aaabbb = end state

14 The Limits of Finite State Technology we can merge the individual FSA for … –ab –aabb –aaabbb aaabbb bb b such direct encoding would require an infinite number of states –and we ’ re using Finite State Automata quite different from the infinity obtained by looping –freely iterate (no counting)

15 The Limits of Finite State Technology example –L = a + b + = { ab, abb, aab, aabb, aaab, abbb, … } –“ one or more a ’ s followed by one or more b ’ s ” Note: –can be divided into two independent halves –each half can be replaced by iteration s1s1 s2s2 s3s3 ba s1s1 s2s2 s3s3 aa s4s4 b s1s1 s2s2 s3s3 ba s4s4 b s1s1 s2s2 s3s3 aa s4s4 b s5s5 b s1s1 s2s2 s3s3 aa s4s4 a s5s5 b s1s1 s2s2 s3s3 ba s4s4 b s5s5 b

16 The Limits of Finite State Technology example –L = a + b + = { ab, abb, aab, aabb, aaab, abbb, … } –“ one or more a ’ s followed by one or more b ’ s ” Note: –can be divided into two independent halves –each half can be replaced by iteration s1s1 s2s2 s3s3 ba s1s1 s2s2 s3s3 aa s4s4 b s1s1 s2s2 s3s3 ba s4s4 b s1s1 s2s2 s3s3 aa s4s4 b s5s5 b s1s1 s2s2 s3s3 aa s4s4 a s5s5 b s1s1 s2s2 s3s3 ba s4s4 b s5s5 b s1s1 s2s2 s3s3 ba s4s4 b s1s1 s2s2 s3s3 aa s4s4 b s5s5 b s0s0 ε ε s1s1 s2s2 s3s3 aa s4s4 a s5s5 b s6s6 b s0s0 ε ε s1s1 s2s2 s3s3 aa s4s4 a s5s5 b s6s6 bb s7s7 s1s1 s2s2 s3s3 aa s4s4 a s5s5 b b s3s3 s4s4 a s5s5 b b a

17 Distance and Finite State Limits Does linear distance between dependent elements create machines with impractically large number of states? Subject-Verb number agreement –The men were... –The man was... –The men forced from their homes were... –The man forced from his home was... (Miller and Chomsky 1963) –The people who called and wanted to rent your house when you go away next year are from California –need “memory” –14 words separating people from are

18 Distance and Finite State Limits (Miller and Chomsky 1963) –The people/person who called and wanted to rent your house when you go away next year are/is from California –need “memory” –14 words separating people from are from (Coleman 2005)

19 Next Time Back to writing grammars... –building sentence meaning –next step towards our translator