Probabilistic Suffix Trees Maria Cutumisu CMPUT 606 October 13, 2004

2 Goal  Provide efficient prediction for protein families  Probabilistic Suffix Trees (PSTs) are variable length Markov models (VMMs)

3 Conceptual Map Probabilistic Suffix Trees ePST Suffix Trees Variable Length Markov Model bPST

4 Background  PSTs were introduced by Ron, Singer, Tishby  Bejerano, Yona made further improvements (bPST)  Poulin – efficient PSTs (ePSTs)  PSTs a.k.a. prediction suffix trees

5 Higher Order Markov Models  A k-order Markov chain: history of length k for conditional probabilities  Exponential storage requirements  Order of the chain increases, amount of training data increases to improve estimation accuracy

6 Variable Length Markov Models (VMMs)  Space and parameter-estimation efficient variable length of the history sequence for prediction only needed parameters are stored  Created from less training data >T1 Test sequence AHGSGYMNAB Training sequences Is T1 in the training set?

7 VMMs  P(sequence) = product of the probabilities of each amino acid given those that precede it  Conditional probability based on the context of each amino acid  A context function k(·) can select the history length based on the context x 1... x i−1 x i  VMMs were first introduced as PSTs

8 PSTs  VMMs for efficient prediction  Pruned during training to contain only required parameters  bPST: represents histories  ePST: represents sequences

9 bPST  Used to represent the histories for prediction instead of the training sequences  The possible histories are the reversed strings of all the substrings of the training sequences

10 Prediction with bPSTs  The conditional probabilities P(x i |x i-1 …) are obtained for each position by tracing a path from the root that matches the preceding residues

11 Construction bPST  We add histories for the training data  Nodes: parameters that estimate the conditional probabilities γ history (a) = P(a|history) P bPST (x i |x i−1,..., x 1 ) = γ x1...xi−1 (x i ) if in bPST else γ x2...xi−1 (x i ) if in bPST etc. else γ (x i )

12 bPST created and pruned using P(01001) = P(0)P(1|0)P(0|01)P(0|010)P(1|0100) = γ (0) γ 0 (1) γ 01 (0) γ 0 * (0) γ 00 * (1) = (13/27)(8/13)(5/8)(5/13)(4/5) = 10400/ = Brett Poulin

13 Complexity bPST  bPST building process requires O(Ln 2 ) time L is the length limit of the tree n is the total length of the training set.  bPST building requires all training sequences at once (in order to get all the reverse substrings) and cannot be done online (the bPST cannot be built as the training data is encountered)  Prediction: O(mL), m = sequence length

14 Improved bPST  Idea: tree with training sequences  n length of all training sequences  m length of tested sequence  Result (theoretical): linear time building O(n) linear time prediction O(m).

15 Efficient PST (ePST)  Used for predicting protein function  ePST represents sequences  Linear construction and prediction

16 Example ePST Brett Poulin

17 Prediction with ePSTs  The probabilities for a substring are obtained for each position by tracing the path representing the sequence from the root  If the entire sequence is not found in the tree, suffix links are followed

18 Construction ePST  ePSTs gain efficiency by representing the training sequences in the PST  Nodes store counts of the subsequence occurrences in the training data (with respect to the complete tree)  Conditional probabilities deducted from the counts are stored as well

19 Example ePST - AYYYA Brett Poulin

20 Complexity ePST  Linear time and space with regards to the combined length of the training sequences O(n)  Linear prediction time O(m)

21 Advantages and Disadvantages  Avoid exponential space requirements and parameter estimation problems of higher order Markov chains  Pruned during training to contain only required parameters  bPSTs for local predictions: more accurate prediction than global  Loss in classification performance: Pfarm, SCOP

22 Conclusions  PSTs require less training and prediction time than HMMs  Despite some loss in classification performance, PSTs compete with HMMs due to PSTs reduced resource demands  PSTs take advantage of VMMs higher order correlations

23 References  Brett Poulin, Sequence-based Protein Function Prediction, Master Thesis, University of Alberta, 2004  G Bejerano, G Yona, Modeling protein families using probabilistic suffix trees, RECOMB’99  G Bejerano, Algorithms for variable length markov chain modeling, Bioinformatics Applications Note, 20(5):788–789, 2004

24 PSTs and HMMs  “HMMs do not capture any higher-order correlations. An HMM assumes that the identity of a particular position is independent of the identity of all other positions.” [1]  PSTs are variable length Markov models for efficient prediction. The prediction uses the longest available context matching the history of the current amino acid.  For protein prediction in general, “the main advantage of PSTs over HMMs is that the training and prediction time requirements of PSTs are much less than for the equivalent HMMs.” [1]

25 Suffix Trees (ST) Brett Poulin

26 bPST  Histories added to the tree must occur more frequently than a threshold P min  The substrings are added in order of length from smallest to largest

27 bPST vs ST  The string s is only added to the tree if the resulting conditional probability at the node to be created will be greater than the minimum prediction probability γ min + α and the probability for the prefix of the string is different (with some ratio r) from the probability assigned to the next shortest substring suf(s) (which is already in the tree). After all the substrings are added to the tree, the probabilities are smoothed according to the parameter γ min.  The smoothing (as calculated by the equation below) prevents any probability from being less than γ min

28 New!