Let us consider a sample word graph as described below: This series of slides provide detailed description of the algorithm used to determine the confidence measure of the words in the hypothesis through word graphs. The core computation is the forward-backward algorithm used to determine the link posterior probabilities. Let us consider a sample word graph as described below: 3/6 2/6 4/6 1/6 5/6 Sil This is a test sentence this the guest quest sense The values on the links are the likelihoods.

Using forward-backward algorithm for determining the link probability. The equations used to compute the alphas and betas are as follows: Computing alphas: Step 1: Initialization: In a conventional HMM forward-backward algorithm we would perform the following – We need to use a slightly modified version of the above equation for processing a word graph. The emission probability will be the language model probability and the initial probability in this case has been taken as 0.01 (assuming we have 100 words in a loop grammar and hence all the words are equally probable with probability 1/100).

The α for the first node in the word graph is computed as follows: Step 2: Induction This step is the main reason we use forward-backward algorithm for computing such probabilities. The alpha values computed in the previous step is used to compute the alphas for the succeeding nodes. Note: Unlike in HMMs where we move from left to right at fixed intervals of time, over here we move from one start time of a word to the next closest word’s start time.

Let us see the computation of the alphas from node 2, the alpha for node 1 was computed in the previous step during initialization. Node 2: Node 3: Node 4: The alpha calculation continues in this manner for all the remaining nodes

Once we compute the alphas using the forward algorithm we begin the beta computation using the backward algorithm. The backward algorithm is similar to the forward algorithm, but we start from the last node and proceed from right to left. Step 1 : Initialization Step 2: Induction

Let us see the computation of the beta values from node 14 and backwards.

Node 11: In a similar manner we obtain the beta values for all the nodes till node 1. We can compute the probabilities on the links (between two nodes) as follows: Let us call this link probability as Γ. Therefore Γ(t-1,t) is computed as the product of α(t-1)*ß(t). These values give the un-normalized posterior probabilities of the word on the link considering all possible paths through the link.

This is the word graph with every node with its corresponding alpha and beta value. quest 1/6 a sense 14 1/6 the 1/6 α =1E-04 β=2.8843E-16 1/6 8 Sil is 2/6 4 1/6 guest 11 3/6 sentence Sil 2/6 1/6 5/6 Sil 6 This 2/6 1 3 4/6 13 15 is 9 4/6 3/6 α=1.861E-14 β=2.776E-8 α=2.886E-20 β=1 3/6 the Sil 5 sentence α=3.438E-18 β=8.33E-3 2/6 this 1/6 α=3.35E-9 β=8.534E-12 is test 2 a 4/6 4/6 12 α =5E-07 β=2.87E-16 7 10 α=4.964E-16 β=5.555E-5 α=7.446E-14 β=3.703E-7 α=1.117E-11 β=2.514E-9 Assumption here is that the probability of occurrence of any word is 0.01. i.e. we have 100 words in a loop grammar

The following word graph shows the links with their corresponding link posterior probabilities (not yet normalized). Γ=1.292E-19 Γ=4.649E-19 Γ=1.292E-19 quest 1/6 Γ=7.749E-20 a sense 14 1/6 the 1/6 Γ=7.749E-20 1/6 8 Sil Γ=4.649E-19 is 2/6 Γ=5.74E-18 4 1/6 guest 11 3/6 sentence Sil 2/6 Sil 6 1/6 5/6 Γ=6.46E-19 This 2/6 1 4/6 13 3 15 is Γ=1.549E-19 9 Γ=3.1E-19 4/6 3/6 3/6 the Γ=3.438E-18 Γ=2.87E-20 Sil Γ=4.288E-18 5 sentence 2/6 this 1/6 Γ=3.1E-19 is test Γ=2.87E-20 2 a 4/6 4/6 12 7 10 Γ=4.136E-18 Γ=8.421E-18 Γ=4.136E-18 Γ=4.136E1-18