1. CS621/CS449 Artificial Intelligence Lecture Notes
Set 8 : 27/10/2004
13/08/2004
CS-621/CS-449 Lecture Notes

2. Outline
Probabilistic Spell Checker (continued from Noisy Channel Model)
Confusion Matrix
27/10/2004
CS-621/CS-449 Lecture Notes

3. Probabilistic Spell Checker Noisy Channel Model
The problem formulation for spell checker is based on the Noisy Channel Model
w t
(wn, wn-1, … , w1) (tm, tm-1, … , t1)
Given t, find the most probable w : Find that ŵ for which P(w|t) is maximum, where t, w and ŵ are strings:
Noisy Channel ŵ
Guess at the correct word
Correct word
Wrongly spelt word
27/10/2004
CS-621/CS-449 Lecture Notes

4. Probabilistic Spell checker
Applying Bayes rule,
Why apply Bayes rule?
Finding p(w|t) Vs p(t|w) ?
P(w|t) or P(t|w) have to be computed by counting c(w,t) or c(t,w) and then normalizing them
Assumptions :
t is obtained from w by a single error of the above type.
The words consist of only alphabets ŵ
27/10/2004
CS-621/CS-449 Lecture Notes

5. Confusion Matrix Confusion Matrix: 26x26
Data structure to store c(a,b)
Different matrices for insertion, deletion, substitution and transposition
Substitution
The number of instances in which a is wrongly substituted by b in the training corpus (denoted sub(x,y) )
27/10/2004
CS-621/CS-449 Lecture Notes

6. Confusion Matrix Insertion
The number of times a letter y is inserted after x wrongly( denoted ins(x,y) )
Transposition
The number of times xy is wrongly transposed to yx ( denoted trans(x,y) )
Deletion
The number of times y is deleted wrongly after x ( denoted del(x,y) )
27/10/2004
CS-621/CS-449 Lecture Notes

7. Confusion Matrix If x and y are alphabets,
sub(x,y) = # times y is written for x (substitution)
ins(x,y) = # times x is written as xy
del(x,y) = # times xy is written as x
trans(x,y) = # times xy is written as yx
27/10/2004
CS-621/CS-449 Lecture Notes

8. Probabilities P(t|w) = P(t|w)S + P(t|w)I + P(t|w)D + P(t|w)X Where
P(t|w)S = sub(x,y) / count of x
P(t|w)I = ins(x,y) / count of x
P(t|w)D = del(x,y) / count of x
P(t|w)X = trans(x,y) / count of x
These are considered to be mutually exclusive events
27/10/2004
CS-621/CS-449 Lecture Notes

9. Example Correct document has ws Wrong document has ts P(maple|aple) =
# (maple was wanted instead of aple) / # (aple)
P(apple|aple) and P(applet|aple) calculated similarly
Leads to problems due to data sparsity.
Hence, use Bayes rule.
27/10/2004
CS-621/CS-449 Lecture Notes