Fall 2004COMP 3351 Simplifications of Context-Free Grammars
Fall 2004COMP 3352 A Substitution Rule Substitute Equivalent grammar
Fall 2004COMP 3353 A Substitution Rule Equivalent grammar Substitute
Fall 2004COMP 3354 In general: Substitute equivalent grammar
Fall 2004COMP 3355 Nullable Variables Nullable Variable:
Fall 2004COMP 3356 Removing Nullable Variables Example Grammar: Nullable variable
Fall 2004COMP 3357 Substitute Final Grammar
Fall 2004COMP 3358 Unit-Productions Unit Production: (a single variable in both sides)
Fall 2004COMP 3359 Removing Unit Productions Observation: Is removed immediately
Fall 2004COMP Example Grammar:
Fall 2004COMP Substitute
Fall 2004COMP Remove
Fall 2004COMP Substitute
Fall 2004COMP Remove repeated productions Final grammar
Fall 2004COMP Useless Productions Some derivations never terminate... Useless Production
Fall 2004COMP Another grammar: Not reachable from S Useless Production
Fall 2004COMP In general: if then variable is useful otherwise, variable is useless contains only terminals
Fall 2004COMP A production is useless if any of its variables is useless Productions useless Variables useless
Fall 2004COMP Removing Useless Productions Example Grammar:
Fall 2004COMP First: find all variables that can produce strings with only terminals Round 1: Round 2:
Fall 2004COMP Keep only the variables that produce terminal symbols: (the rest variables are useless) Remove useless productions
Fall 2004COMP Second: Find all variables reachable from Use a Dependency Graph not reachable
Fall 2004COMP Keep only the variables reachable from S Final Grammar (the rest variables are useless) Remove useless productions
Fall 2004COMP Removing All Step 1: Remove -productions Step 2: Remove Unit-productions Step 3: Remove Useless productions
Fall 2004COMP Normal Forms for Context-free Grammars
Fall 2004COMP Chomsky Normal Form Each productions has form: variable or terminal
Fall 2004COMP Examples: Not Chomsky Normal Form Chomsky Normal Form
Fall 2004COMP Conversion to Chomsky Normal Form Example: Not Chomsky Normal Form
Fall 2004COMP Introduce variables for terminals:
Fall 2004COMP Introduce intermediate variable:
Fall 2004COMP Introduce intermediate variable:
Fall 2004COMP Final grammar in Chomsky Normal Form: Initial grammar
Fall 2004COMP From any context-free grammar (which doesn’t produce ) not in Chomsky Normal Form we can obtain: An equivalent grammar in Chomsky Normal Form In general:
Fall 2004COMP The Procedure First remove: Nullable variables Unit productions
Fall 2004COMP Then, for every symbol : In productions: replace with Add production New variable:
Fall 2004COMP Replace any production with New intermediate variables:
Fall 2004COMP Theorem: For any context-free grammar (which doesn’t produce ) there is an equivalent grammar in Chomsky Normal Form
Fall 2004COMP Observations Chomsky normal forms are good for parsing and proving theorems It is very easy to find the Chomsky normal form for any context-free grammar
Fall 2004COMP Greibach Normal Form All productions have form: symbolvariables
Fall 2004COMP Examples: Greibach Normal Form Not Greibach Normal Form
Fall 2004COMP Conversion to Greibach Normal Form: Greibach Normal Form
Fall 2004COMP Theorem: For any context-free grammar (which doesn’t produce ) there is an equivalent grammar in Greibach Normal Form
Fall 2004COMP Observations Greibach normal forms are very good for parsing It is hard to find the Greibach normal form of any context-free grammar
Fall 2004COMP The CYK Parser
Fall 2004COMP The CYK Membership Algorithm Input: Grammar in Chomsky Normal Form String Output: find if
Fall 2004COMP The Algorithm Grammar : String : Input example:
Fall 2004COMP 33547
Fall 2004COMP 33548
Fall 2004COMP 33549
Fall 2004COMP 33550
Fall 2004COMP Therefore: Time Complexity: The CYK algorithm can be easily converted to a parser (bottom up parser) Observation: