1. Prof. Pushpak Bhattacharyya, IIT Bombay
CS460/IT632 Natural Language Processing/Language Technology for the Web Lecture 20 (28/03/06) Prof. Pushpak Bhattacharyya IIT Bombay Probabilistic Parsing
28/03/06
Prof. Pushpak Bhattacharyya, IIT Bombay

2. Example Grammar – Syntax Items
S  NP VP 0.8
 VP 0.2
NP  n 0.4
 n PP 0.4
 n NP 0.2
VP  v 0.3
 v NP 0.3
 v PP 0.2
 v NP PP 0.2
PP  p NP 1.0
28/03/06
Prof. Pushpak Bhattacharyya, IIT Bombay

3. Example Grammar - Lexical Items
p  like 1.0
v  swat 0.2
 flies 0.4
 like 0.4
n  swat 0.05
 flies 0.45
 ants 0.5
28/03/06
Prof. Pushpak Bhattacharyya, IIT Bombay

4. Prof. Pushpak Bhattacharyya, IIT Bombay
Language Model
Language Model can be categorized into two types –
N-grams
Probabilistic context free grammar (PCFG)
A Language Model either accepts or generates a piece of text.
28/03/06
Prof. Pushpak Bhattacharyya, IIT Bombay

5. Input Sentence & Interpretations
Let the input sentence be –
“Swat flies like ants”
(in the sense like – “Time flies like an arrow”)
Swat means –
n : a violent blow
v : to render a violent blow
Interpretations –
Swat is the subject of fly.
Swat is a qualifier for fly (some kind of fly).
Swat is a verb with ‘fly’ as an object.
28/03/06
Prof. Pushpak Bhattacharyya, IIT Bombay

6. Parse Tree for Interpretation-1NP VP v
ants PP p
flies
like
swat
28/03/06
Prof. Pushpak Bhattacharyya, IIT Bombay

7. Parse Tree for Interpretation-2NP VP v
ants
like
swat
flies
28/03/06
Prof. Pushpak Bhattacharyya, IIT Bombay

8. Parse Tree for Interpretation-3PP VP v
flies NP
swat n
like
ants
28/03/06
Prof. Pushpak Bhattacharyya, IIT Bombay

9. Prof. Pushpak Bhattacharyya, IIT Bombay
Comparison – CFG & PCFG
CFG
Corpus insensitive
Does not deal with ambiguity effectively
Fails in case of noise
Does not scale up
PCFG
Corpus sensitive
Principled way of dealing with ambiguity
Deals with noise
Scales up (have good learning algorithms)
28/03/06
Prof. Pushpak Bhattacharyya, IIT Bombay

10. Meaning of the Probability of a Rule
P( A  α : p )
Probability of the rule
A  α is p
Definition- We say that in a derivation sequence D, Nj dominates tkl
where w1 w2 …… wn
tkl is a segment of the sequence w1, w2,…wn
iff
tkl is generated (directly or indirectly) from Nj by a set of application of production rules.
28/03/06
Prof. Pushpak Bhattacharyya, IIT Bombay

11. Meaning of the Probability of a Rule (Contd.)
Then the probability of a parse tree t given the sentence s ( = w1 , w2 ,…wn) is defined as
P(w1 w2 … wn | s) =
P(X1k1, Xk1k2,… Xkm,n , w1, w2, … wn | s1,n)
28/03/06
Prof. Pushpak Bhattacharyya, IIT Bombay