1. Fall 2004COMP 3351 Simplifications of Context-Free Grammars

2. Fall 2004COMP 3352 A Substitution Rule Substitute Equivalent grammar

3. Fall 2004COMP 3353 A Substitution Rule Equivalent grammar Substitute

4. Fall 2004COMP 3354 In general: Substitute equivalent grammar

5. Fall 2004COMP 3355 Nullable Variables Nullable Variable:

6. Fall 2004COMP 3356 Removing Nullable Variables Example Grammar: Nullable variable

7. Fall 2004COMP 3357 Substitute Final Grammar

8. Fall 2004COMP 3358 Unit-Productions Unit Production: (a single variable in both sides)

9. Fall 2004COMP 3359 Removing Unit Productions Observation: Is removed immediately

10. Fall 2004COMP 33510 Example Grammar:

11. Fall 2004COMP 33511 Substitute

12. Fall 2004COMP 33512 Remove

13. Fall 2004COMP 33513 Substitute

14. Fall 2004COMP 33514 Remove repeated productions Final grammar

15. Fall 2004COMP 33515 Useless Productions Some derivations never terminate... Useless Production

16. Fall 2004COMP 33516 Another grammar: Not reachable from S Useless Production

17. Fall 2004COMP 33517 In general: if then variable is useful otherwise, variable is useless contains only terminals

18. Fall 2004COMP 33518 A production is useless if any of its variables is useless Productions useless Variables useless

19. Fall 2004COMP 33519 Removing Useless Productions Example Grammar:

20. Fall 2004COMP 33520 First: find all variables that can produce strings with only terminals Round 1: Round 2:

21. Fall 2004COMP 33521 Keep only the variables that produce terminal symbols: (the rest variables are useless) Remove useless productions

22. Fall 2004COMP 33522 Second: Find all variables reachable from Use a Dependency Graph not reachable

23. Fall 2004COMP 33523 Keep only the variables reachable from S Final Grammar (the rest variables are useless) Remove useless productions

24. Fall 2004COMP 33524 Removing All Step 1: Remove -productions Step 2: Remove Unit-productions Step 3: Remove Useless productions

25. Fall 2004COMP 33525 Normal Forms for Context-free Grammars

26. Fall 2004COMP 33526 Chomsky Normal Form Each productions has form: variable or terminal

27. Fall 2004COMP 33527 Examples: Not Chomsky Normal Form Chomsky Normal Form

28. Fall 2004COMP 33528 Conversion to Chomsky Normal Form Example: Not Chomsky Normal Form

29. Fall 2004COMP 33529 Introduce variables for terminals:

30. Fall 2004COMP 33530 Introduce intermediate variable:

31. Fall 2004COMP 33531 Introduce intermediate variable:

32. Fall 2004COMP 33532 Final grammar in Chomsky Normal Form: Initial grammar

33. Fall 2004COMP 33533 From any context-free grammar (which doesn’t produce ) not in Chomsky Normal Form we can obtain: An equivalent grammar in Chomsky Normal Form In general:

34. Fall 2004COMP 33534 The Procedure First remove: Nullable variables Unit productions

35. Fall 2004COMP 33535 Then, for every symbol : In productions: replace with Add production New variable:

36. Fall 2004COMP 33536 Replace any production with New intermediate variables:

37. Fall 2004COMP 33537 Theorem: For any context-free grammar (which doesn’t produce ) there is an equivalent grammar in Chomsky Normal Form

38. Fall 2004COMP 33538 Observations Chomsky normal forms are good for parsing and proving theorems It is very easy to find the Chomsky normal form for any context-free grammar

39. Fall 2004COMP 33539 Greibach Normal Form All productions have form: symbolvariables

40. Fall 2004COMP 33540 Examples: Greibach Normal Form Not Greibach Normal Form

41. Fall 2004COMP 33541 Conversion to Greibach Normal Form: Greibach Normal Form

42. Fall 2004COMP 33542 Theorem: For any context-free grammar (which doesn’t produce ) there is an equivalent grammar in Greibach Normal Form

43. Fall 2004COMP 33543 Observations Greibach normal forms are very good for parsing It is hard to find the Greibach normal form of any context-free grammar

44. Fall 2004COMP 33544 The CYK Parser

45. Fall 2004COMP 33545 The CYK Membership Algorithm Input: Grammar in Chomsky Normal Form String Output: find if

46. Fall 2004COMP 33546 The Algorithm Grammar : String : Input example:

47. Fall 2004COMP 33547

48. Fall 2004COMP 33548

49. Fall 2004COMP 33549

50. Fall 2004COMP 33550

51. Fall 2004COMP 33551 Therefore: Time Complexity: The CYK algorithm can be easily converted to a parser (bottom up parser) Observation: