1. LING 388 Language and Computers Lecture 11 10/7/03 Sandiway FONG

2. Administrivia Computer Lab Computer Lab  Double Class:  Thursday 9th and Tuesday 14th  Location:  SBS 224 TA Office Hours TA Office Hours  Change in time and location  Now Tuesdays after class 12:15 pm - 1:15 pm  SBS 224

3. Review Chomsky Hierarchy: Chomsky Hierarchy:  Type-0 General rewrite rules  Type-1 Context-sensitive rules  a n b n c n  Implementation: type-2 rules + counter Type-2 Context-free rulesType-2 Context-free rules a n b n a n b n Implementation: type-2 rules or type-3 rules + counter Implementation: type-2 rules or type-3 rules + counter –Type-3 Regular grammar rules – a + b + – Implementation: type-3 rules

4. Review Production rule formats: Production rule formats:  Type-1:  Next slide…  Type-2: A -> A ->  Type-3:Type-3: –A -> Bc A -> c or –A -> cB A -> c where … where …  A  V N, c  V T and  (V N u V T ) *

5. Context-sensitive Grammars Type-2 and 3 grammars may only have a single non- terminal on the left. Type-2 and 3 grammars may only have a single non- terminal on the left. Type-1 (context-sensitive) grammars extend what’s allowed on the left. Type-1 (context-sensitive) grammars extend what’s allowed on the left.  Production rules have the format:   ->  such that |  | >= |  |  where …   (V N u V T ) + Note:  should not be comprised of just terminal symbols   (V N u V T ) * Notes: Notes:  Length constraint means a rule like A -> is not permitted

6. Context-sensitive Grammars (Almost equivalent) alternative definition: (Almost equivalent) alternative definition:  Production rules have the format:   ->   where …   V N   (V N u V T ) * Notes: Notes:   and  are “copied” over from the left to the right side unchanged   constitutes the (left and right) contexts for non-terminal A  i.e. A ->  in context  However, However,  … will be more convenient (for our purposes) to use the first definition

7. Context-sensitive Grammars Equivalence (informal) Equivalence (informal)  Can transform a context-sensitive rule of form:  AB -> CD where A,B,C,D are all different non-terminalswhere A,B,C,D are all different non-terminals into rules respecting the form:   ->  Invent non-terminals A’ and B’ Invent non-terminals A’ and B’ Conversion: Conversion:  AB -> A’B(left context  empty, right context  = B)  A’B -> A’B’(left context  = A’, right context  empty)  A’B’ -> CB’(left context  empty, right context  = B’)  CB’ -> CD(left context  = C, right context  empty)  Note:  All four rules respect  ->  yet clearly AB => + CD

8. Context-sensitive Grammars Example: Example:  G abc is a type-1 grammar such that L(G abc ) = {a n b n c n | n >=1 }  G abc has 4 production rules:  S -> aSBc  S -> abc  cB -> Bc  bB -> bb  Notes:  1st two rules context-free  3rd/4th rules context-sensitive c (resp. b) left context for non-terminal B in cB -> Bc (bB -> bb)c (resp. b) left context for non-terminal B in cB -> Bc (bB -> bb)  Length restriction is respected

9. Context-sensitive Grammars Compare G abc to DCG (based on type-2 rules) shown in Lecture 10: Compare G abc to DCG (based on type-2 rules) shown in Lecture 10:  s --> [a],t(1),[c].  t(N) --> [a],{M is N+1},t(M),[c].  t(N) --> u(N).  u(N) --> {N > 1}, [b],{M is N-1},u(M).  u(1) --> [b].

10. Context-sensitive Grammars How does G abc work? How does G abc work? Basic Idea: Basic Idea:  G abc has two stages: 1. Build an equal number of as, bs and cs 2. Re-arrange them into the correct linear order We have 4 production rules: We have 4 production rules:  S -> aSBcStage 1: Build  S -> abc  cB -> BcStage 2: Re-arrange  bB -> bb

11. Context-sensitive Grammars Build Stage: Build Stage:  S -> aSBc  S -> abc Sentential Forms: Sentential Forms:  S  aSBcUsing 1st rule  aaSBcBcUsing 1st rule  aaabcBcBcUsing 2nd rule Notes: Notes:  same number of as, bs (counting b and B together) and cs at each sentential form

12. Context-sensitive Grammars Re-arrangement Stage: Re-arrangement Stage:  cB -> Bc  bB -> bb Example Sentential Form: Example Sentential Form:  aaabcBcBc Question: What can we do? Question: What can we do? Answer: Answer:  Basic Idea 1: Re-arrange the order of Bs and cs  want to move the Bs to the left  want to move the cs to the right Question: How do we know when to stop re-arranging? Question: How do we know when to stop re-arranging? Answer: Answer:  Basic Idea 2: Stop when a B comes into contact with a b

13. Context-sensitive Grammars Re-arrangement Stage: Re-arrangement Stage:  cB -> Bcre-arrange c and B  bB -> bb stopping condition Example Sentential Form: Example Sentential Form:  aaabcBcBc Basic Idea 1: Re-arrange the order of Bs and cs Basic Idea 1: Re-arrange the order of Bs and cs  cB -> Bc  “If a c precedes a B, the order is wrong, let’s flip them” Example of Derivation: Example of Derivation:  aaabcBcBc  aaabBccBc  aaabBcBcc  aaabBBccc

14. Context-sensitive Grammars Re-arrangement Stage: Re-arrangement Stage:  cB -> Bcre-arrange c and B  bB -> bb stopping condition Example Sentential Form: Example Sentential Form:  aaabBBccc We still have non-terminals in the sentential string We still have non-terminals in the sentential string  so we’re not done yet Apply stopping condition: Apply stopping condition:  bB -> bb Example Derivation: Example Derivation:  aaabBBccc  aaabbBccc  aaabbbccc(a 3 b 3 c 3 ) Final sentential form contains no non-terminals, so we’re done! Final sentential form contains no non-terminals, so we’re done!

15. Grammar Rule Implementations Chomsky Hierarchy: Chomsky Hierarchy:  Type-0 General rewrite rules  Type-1 Context-sensitive rules  a n b n c n  Implementation: type-2 rules + counter or type-1 rules Type-2 Context-free rulesType-2 Context-free rules a n b n a n b n Implementation: type-2 rules or type-3 rules + counter Implementation: type-2 rules or type-3 rules + counter –Type-3 Regular grammar rules – a + b + – Implementation: type-3 rules

16. Type-0 Grammars General rewrite rule system General rewrite rule system Rule format: Rule format:   ->   where   (V N u V T ) +   (V N u V T ) *

17. Equivalent Automata Implementations Chomsky Hierarchy: Chomsky Hierarchy:  Type-0 General rewrite rules  Implementation: Turing Machine (TM)  Type-1 Context-sensitive rules  a n b n c n  Implementation: Linear Bounded Automata (LBA) Type-2 Context-free rulesType-2 Context-free rules a n b n a n b n Implementation: Non-deterministic Push-Down Automata (NPDA) Implementation: Non-deterministic Push-Down Automata (NPDA) –Type-3 Regular grammar rules – a + b + – Implementation: Finite State Automata (FSA)

18. Equivalent Automata Implementations Machine Characteristics: Machine Characteristics:  All the machine types have a finite state core  However, they differ with respect to working memory  The difference in expressive power can be traced to limitations on the working memory…

19. Equivalent Automata Implementations Grammar Working Memory Notes Type-3None Type-2Stack Push/Pop operations No limit on size of stack Type-1Tape Write/(Erase)/Read Move (read head) left/right Length limited to a linear function of the input length Type-0Tape Write/(Erase)/Read Move (read head) left/right Unlimited length

20. DCG and Context-sensitive Grammars DCG formalism is based on type-2 (context-free) grammars DCG formalism is based on type-2 (context-free) grammars It is powerful enough to encode type-1 (context- sensitive) grammars It is powerful enough to encode type-1 (context- sensitive) grammars and beyond…  Example:  in Lecture 10, we exhibited a DCG for a n b n c n … based on type-2 rules plus a counter… based on type-2 rules plus a counter

21. DCG and Chomsky Hierarchy Regular Grammars = Type-3 FSA Regular Expressions DCG = Type-0 Type-2 Type-1

22. DCG and Context-sensitive Grammars However, we cannot write DCG rules for context-sensitive rules directly However, we cannot write DCG rules for context-sensitive rules directly  Example:  can’t write [c],b --> b, [c].  for cB -> Bc Note: Note:  we can write Prolog code that takes a context-sensitive grammar and interprets it  Possible term programming project?

23. This concludes our tour of the grammar hierarchy for this course This concludes our tour of the grammar hierarchy for this course But before we leave … But before we leave …

24. Extra Credit Homework Question We now know We now know  {a n b n c n | n >=1 } is a context-sensitive (not context- free) language  {a n b n | n >=1 } is a context-free (not regular) language How about? How about?  L abcd = {a n b n c n d n | n >=1 } Write a grammar for L abcd Write a grammar for L abcd Notes: Notes:  Not a computer lab homework  Extra credit question is entirely optional  A chance to make up some ground if you lost points on Homeworks 1 or 2  Hand in your solution at the same time as Homework 3