Download presentation
Presentation is loading. Please wait.
1
Simplifications of Context-Free Grammars
2
Assumption The language discussed here does not contain the empty string. Let L be any context-free language, let G = (V, T, S, P) be a context-free grammar for L – { }. Then the grammar obtain by adding to V the new variable S0, Making the start variable, and adding to P the productions: generate L.
3
A Substitution Rule Equivalent grammar Substitute
4
A Substitution Rule Substitute Equivalent grammar
5
In general: Substitute equivalent grammar
6
Nullable Variables Nullable Variable:
7
Removing Nullable Variables
Example Grammar: Nullable variable
8
Why? Language generated by the grammar? (ab*)(ab*)*
How many derivation steps for the string aaaa?
9
Language generated by the grammar?
(ab*)(ab*)* How many derivation steps for the string aaaa?
10
Final Grammar Substitute
11
Algorithm to construct the set of nullable variables
Input: context-free grammar G = (V, T, S, P) 1. 2. Repeat 2.1. PREV := NULL 2.2 for each variable do if there is an A rule and , then until NULL = PREV
12
Example Iteration NULL PREV {B, C} 1 {A, B, C} 2
13
Once the set of nullable variables has been found, a new set of production rules P’ is constructed.
Construction: For each production in the grammar G - the production is put into P’ - as well as all those generated by replacing nullable variables with in all possible combinations - if all are nullable, the production is not added
14
Example Nullable variables: A, B, C New Productions:
15
Unit-Productions Unit Production: (a single variable in both sides)
16
Removing Unit Productions
Observation: Is removed immediately
17
How about:
18
Example Grammar:
19
Substitute
20
Remove
21
Substitute
22
Remove repeated productions
Final grammar
23
Algorithm to remove unit productions
Draw a dependency graph with an edge (A, B) whenever the grammar has a unit production The new grammar G’ is generated by first putting into P’ all non-unit productions of P. Then for all A and B satisfying , we add to P’ Where is the set of all rules in P’ with B on the left
24
Another Example Add original non-unit productions:
Use A->a|bc to substitute: Use B->bb to substitute:
25
Useless Productions Useless Production
Some derivations never terminate...
26
Another grammar: Useless Production Not reachable from S
27
then variable is useful
In general: contains only terminals if then variable is useful otherwise, variable is useless
28
A production is useless
if any of its variables is useless Productions useless Variables useless
29
Removing Useless Productions
Example Grammar:
30
First: find all variables that can produce strings with only terminals
Round 1: Round 2:
31
Keep only the variables that produce terminal symbols:
(the rest variables are useless) Remove useless productions
32
Second: Find all variables reachable from Use a Dependency Graph not
33
Keep only the variables reachable from S
(the rest variables are useless) Final Grammar Remove useless productions
34
Removing All Step 1: Remove Nullable Variables Step 2: Remove Unit-Productions Step 3: Remove Useless Variables
35
Normal Forms for Context-free Grammars
36
Chomsky Normal Form Each productions has form: or variable variable
terminal
37
Examples: Chomsky Normal Form Not Chomsky Normal Form
38
Conversion to Chomsky Normal Form
Example: Not Chomsky Normal Form
39
Introduce variables for terminals:
40
Introduce intermediate variable:
41
Introduce intermediate variable:
42
Final grammar in Chomsky Normal Form:
Initial grammar
43
In general: From any context-free grammar (which doesn’t produce ) not in Chomsky Normal Form we can obtain: An equivalent grammar in Chomsky Normal Form
44
The Procedure First remove: Nullable variables Unit productions
Useless productions
45
Then, for every symbol : Add production In productions: replace with New variable:
46
Replace any production
with New intermediate variables:
47
Theorem: For any context-free grammar (which doesn’t produce )
there is an equivalent grammar in Chomsky Normal Form
48
Observations Chomsky normal forms are good
for parsing and proving theorems It is very easy to find the Chomsky normal form for any context-free grammar
49
Greinbach Normal Form All productions have form: symbol variables
50
Examples: Greinbach Normal Form Not Greinbach Normal Form
51
Conversion to Greinbach Normal Form:
52
Theorem: For any context-free grammar (which doesn’t produce )
there is an equivalent grammar in Greinbach Normal Form
53
Observations Greinbach normal forms are very good for parsing
It is hard to find the Greinbach normal form of any context-free grammar
54
Compilers
55
Add v,v,0 cmp v,5 jmplt ELSE THEN: add x, 12,v ELSE: WHILE: cmp x,3 ... Machine Code Program v = 5; if (v>5) x = 12 + v; while (x !=3) { x = x - 3; v = 10; } ...... Compiler
56
Compiler Lexical analyzer parser input output machine code program
57
A parser knows the grammar
of the programming language
58
Parser PROGRAM STMT_LIST STMT_LIST STMT; STMT_LIST | STMT;
STMT EXPR | IF_STMT | WHILE_STMT | { STMT_LIST } EXPR EXPR + EXPR | EXPR - EXPR | ID IF_STMT if (EXPR) then STMT | if (EXPR) then STMT else STMT WHILE_STMT while (EXPR) do STMT
59
The parser finds the derivation
of a particular input derivation Parser input E => E + E => E + E * E => 10 + E*E => * E => * 5 E -> E + E | E * E | INT * 5
60
derivation tree derivation E E => E + E => E + E * E => 10 + E*E => * E => * 5 E + E 10 E E * 2 5
61
derivation tree E machine code E + E mult a, 2, 5 add b, 10, a 10 E E * 2 5
62
Parsing
63
Parser input string derivation grammar
64
Example: Parser derivation input ?
65
Exhaustive Search Phase 1: Find derivation of
All possible derivations of length 1
67
Phase 2 Phase 1
68
Phase 2 Phase 3
69
Final result of exhaustive search
(top-down parsing) Parser input derivation
70
Time complexity of exhaustive search
Suppose there are no productions of the form Number of phases for string :
71
For grammar with rules Time for phase 1: possible derivations
72
Time for phase 2: possible derivations
73
Time for phase : possible derivations
74
Extremely bad!!! Total time needed for string : phase 1 phase 2|w|
75
There exist faster algorithms
for specialized grammars S-grammar: symbol string of variables Pair appears once
76
S-grammar example: Each string has a unique derivation
77
For S-grammars: In the exhaustive search parsing there is only one choice in each phase Time for a phase: Total time for parsing string :
78
For general context-free grammars:
There exists a parsing algorithm that parses a string in time (we will show it in the next class)
79
The CYK Parser
80
The CYK Membership Algorithm
Input: Grammar in Chomsky Normal Form String Output: find if
81
The Algorithm Input example: Grammar : String :
86
Therefore: Time Complexity: Observation: The CYK algorithm can be easily converted to a parser (bottom up parser)
Similar presentations
© 2024 SlidePlayer.com. Inc.
All rights reserved.