LING 388 Language and Computers Lecture 11 10/7/03 Sandiway FONG

Administrivia Computer Lab Computer Lab  Double Class:  Thursday 9th and Tuesday 14th  Location:  SBS 224 TA Office Hours TA Office Hours  Change in time and location  Now Tuesdays after class 12:15 pm - 1:15 pm  SBS 224

Review Chomsky Hierarchy: Chomsky Hierarchy:  Type-0 General rewrite rules  Type-1 Context-sensitive rules  a n b n c n  Implementation: type-2 rules + counter Type-2 Context-free rulesType-2 Context-free rules a n b n a n b n Implementation: type-2 rules or type-3 rules + counter Implementation: type-2 rules or type-3 rules + counter –Type-3 Regular grammar rules – a + b + – Implementation: type-3 rules

Review Production rule formats: Production rule formats:  Type-1:  Next slide…  Type-2: A -> A ->  Type-3:Type-3: –A -> Bc A -> c or –A -> cB A -> c where … where …  A  V N, c  V T and  (V N u V T ) *

Context-sensitive Grammars Type-2 and 3 grammars may only have a single non- terminal on the left. Type-2 and 3 grammars may only have a single non- terminal on the left. Type-1 (context-sensitive) grammars extend what’s allowed on the left. Type-1 (context-sensitive) grammars extend what’s allowed on the left.  Production rules have the format:   ->  such that |  | >= |  |  where …   (V N u V T ) + Note:  should not be comprised of just terminal symbols   (V N u V T ) * Notes: Notes:  Length constraint means a rule like A -> is not permitted

Context-sensitive Grammars (Almost equivalent) alternative definition: (Almost equivalent) alternative definition:  Production rules have the format:   ->   where …   V N   (V N u V T ) * Notes: Notes:   and  are “copied” over from the left to the right side unchanged   constitutes the (left and right) contexts for non-terminal A  i.e. A ->  in context  However, However,  … will be more convenient (for our purposes) to use the first definition

Context-sensitive Grammars Equivalence (informal) Equivalence (informal)  Can transform a context-sensitive rule of form:  AB -> CD where A,B,C,D are all different non-terminalswhere A,B,C,D are all different non-terminals into rules respecting the form:   ->  Invent non-terminals A’ and B’ Invent non-terminals A’ and B’ Conversion: Conversion:  AB -> A’B(left context  empty, right context  = B)  A’B -> A’B’(left context  = A’, right context  empty)  A’B’ -> CB’(left context  empty, right context  = B’)  CB’ -> CD(left context  = C, right context  empty)  Note:  All four rules respect  ->  yet clearly AB => + CD

Context-sensitive Grammars Example: Example:  G abc is a type-1 grammar such that L(G abc ) = {a n b n c n | n >=1 }  G abc has 4 production rules:  S -> aSBc  S -> abc  cB -> Bc  bB -> bb  Notes:  1st two rules context-free  3rd/4th rules context-sensitive c (resp. b) left context for non-terminal B in cB -> Bc (bB -> bb)c (resp. b) left context for non-terminal B in cB -> Bc (bB -> bb)  Length restriction is respected

Context-sensitive Grammars Compare G abc to DCG (based on type-2 rules) shown in Lecture 10: Compare G abc to DCG (based on type-2 rules) shown in Lecture 10:  s --> [a],t(1),[c].  t(N) --> [a],{M is N+1},t(M),[c].  t(N) --> u(N).  u(N) --> {N > 1}, [b],{M is N-1},u(M).  u(1) --> [b].

Context-sensitive Grammars How does G abc work? How does G abc work? Basic Idea: Basic Idea:  G abc has two stages: 1. Build an equal number of as, bs and cs 2. Re-arrange them into the correct linear order We have 4 production rules: We have 4 production rules:  S -> aSBcStage 1: Build  S -> abc  cB -> BcStage 2: Re-arrange  bB -> bb

Context-sensitive Grammars Build Stage: Build Stage:  S -> aSBc  S -> abc Sentential Forms: Sentential Forms:  S  aSBcUsing 1st rule  aaSBcBcUsing 1st rule  aaabcBcBcUsing 2nd rule Notes: Notes:  same number of as, bs (counting b and B together) and cs at each sentential form

Context-sensitive Grammars Re-arrangement Stage: Re-arrangement Stage:  cB -> Bc  bB -> bb Example Sentential Form: Example Sentential Form:  aaabcBcBc Question: What can we do? Question: What can we do? Answer: Answer:  Basic Idea 1: Re-arrange the order of Bs and cs  want to move the Bs to the left  want to move the cs to the right Question: How do we know when to stop re-arranging? Question: How do we know when to stop re-arranging? Answer: Answer:  Basic Idea 2: Stop when a B comes into contact with a b

Context-sensitive Grammars Re-arrangement Stage: Re-arrangement Stage:  cB -> Bcre-arrange c and B  bB -> bb stopping condition Example Sentential Form: Example Sentential Form:  aaabcBcBc Basic Idea 1: Re-arrange the order of Bs and cs Basic Idea 1: Re-arrange the order of Bs and cs  cB -> Bc  “If a c precedes a B, the order is wrong, let’s flip them” Example of Derivation: Example of Derivation:  aaabcBcBc  aaabBccBc  aaabBcBcc  aaabBBccc

Context-sensitive Grammars Re-arrangement Stage: Re-arrangement Stage:  cB -> Bcre-arrange c and B  bB -> bb stopping condition Example Sentential Form: Example Sentential Form:  aaabBBccc We still have non-terminals in the sentential string We still have non-terminals in the sentential string  so we’re not done yet Apply stopping condition: Apply stopping condition:  bB -> bb Example Derivation: Example Derivation:  aaabBBccc  aaabbBccc  aaabbbccc(a 3 b 3 c 3 ) Final sentential form contains no non-terminals, so we’re done! Final sentential form contains no non-terminals, so we’re done!

Grammar Rule Implementations Chomsky Hierarchy: Chomsky Hierarchy:  Type-0 General rewrite rules  Type-1 Context-sensitive rules  a n b n c n  Implementation: type-2 rules + counter or type-1 rules Type-2 Context-free rulesType-2 Context-free rules a n b n a n b n Implementation: type-2 rules or type-3 rules + counter Implementation: type-2 rules or type-3 rules + counter –Type-3 Regular grammar rules – a + b + – Implementation: type-3 rules

Type-0 Grammars General rewrite rule system General rewrite rule system Rule format: Rule format:   ->   where   (V N u V T ) +   (V N u V T ) *

Equivalent Automata Implementations Chomsky Hierarchy: Chomsky Hierarchy:  Type-0 General rewrite rules  Implementation: Turing Machine (TM)  Type-1 Context-sensitive rules  a n b n c n  Implementation: Linear Bounded Automata (LBA) Type-2 Context-free rulesType-2 Context-free rules a n b n a n b n Implementation: Non-deterministic Push-Down Automata (NPDA) Implementation: Non-deterministic Push-Down Automata (NPDA) –Type-3 Regular grammar rules – a + b + – Implementation: Finite State Automata (FSA)

Equivalent Automata Implementations Machine Characteristics: Machine Characteristics:  All the machine types have a finite state core  However, they differ with respect to working memory  The difference in expressive power can be traced to limitations on the working memory…

Equivalent Automata Implementations Grammar Working Memory Notes Type-3None Type-2Stack Push/Pop operations No limit on size of stack Type-1Tape Write/(Erase)/Read Move (read head) left/right Length limited to a linear function of the input length Type-0Tape Write/(Erase)/Read Move (read head) left/right Unlimited length

DCG and Context-sensitive Grammars DCG formalism is based on type-2 (context-free) grammars DCG formalism is based on type-2 (context-free) grammars It is powerful enough to encode type-1 (context- sensitive) grammars It is powerful enough to encode type-1 (context- sensitive) grammars and beyond…  Example:  in Lecture 10, we exhibited a DCG for a n b n c n … based on type-2 rules plus a counter… based on type-2 rules plus a counter

DCG and Chomsky Hierarchy Regular Grammars = Type-3 FSA Regular Expressions DCG = Type-0 Type-2 Type-1

DCG and Context-sensitive Grammars However, we cannot write DCG rules for context-sensitive rules directly However, we cannot write DCG rules for context-sensitive rules directly  Example:  can’t write [c],b --> b, [c].  for cB -> Bc Note: Note:  we can write Prolog code that takes a context-sensitive grammar and interprets it  Possible term programming project?

This concludes our tour of the grammar hierarchy for this course This concludes our tour of the grammar hierarchy for this course But before we leave … But before we leave …

Extra Credit Homework Question We now know We now know  {a n b n c n | n >=1 } is a context-sensitive (not context- free) language  {a n b n | n >=1 } is a context-free (not regular) language How about? How about?  L abcd = {a n b n c n d n | n >=1 } Write a grammar for L abcd Write a grammar for L abcd Notes: Notes:  Not a computer lab homework  Extra credit question is entirely optional  A chance to make up some ground if you lost points on Homeworks 1 or 2  Hand in your solution at the same time as Homework 3