1. CS 5263 & CS 4593 Bioinformatics Exact String Matching Algorithms

2. Overview Sequence alignment: two sub-problems: –How to score an alignment with errors –How to find an alignment with the best score Today: exact string matching –Does not allow any errors –Efficiency becomes the sole consideration Time and space

3. Why exact string matching? The most fundamental string comparison problem Often the core of more complex string comparison algorithms –E.g., BLAST Often repeatedly called by other methods –Usually the most time consuming part –Small improvement could improve overall efficiency considerably

4. Definitions Text: a longer string T (length m) Pattern: a shorter string P (length n) Exact matching: find all occurrences of P in T T P length m length n

5. The naïve algorithm

6. Time complexity Worst case: O(mn) How to speedup? –Pre-processing T or P –Why pre-processing can save us time? Uncovers the structure of T or P Determines when we can skip ahead without missing anything Determines when we can infer the result of character comparisons without doing them.

7. Cost for exact string matching Total cost = cost (preprocessing) + cost(comparison) + cost(output) Constant Minimize Overhead Hope: gain > overhead

8. String matching scenarios One T and one P –Search a word in a document One T and many P all at once –Search a set of words in a document –Spell checking (fixed P) One fixed T, many P –Search a completed genome for short sequences Two (or many) T’s for common patterns Q: Which one to pre-process? A: Always pre-process the shorter seq, or the one that is repeatedly used

9. Pre-processing algs Pattern preprocessing –Knuth-Morris-Pratt algorithm (KMP) –Aho-Corasick algorithm Multiple patterns –Boyer – Moore algorithm (discuss if have time) The choice of most cases Typically sub-linear time Text preprocessing –Suffix tree Very useful for many purposes –Suffix array –Burrows-Wheeler Transformation

10. Algorithm KMP: Intuitive example 1 Observation: by reasoning on the pattern alone, we can determine that if a mismatch happened when comparing P[8] with T[i], we can shift P by four chars, and compare P[4] with T[i], without missing any possible matches. Number of comparisons saved: 6 abcxabc T abcxabcde P mismatch abcxabc T abcxabcde Naïve approach: abcxabcde ?

11. ? Intuitive example 2 Observation: by reasoning on the pattern alone, we can determine that if a mismatch happened between P[7] and T[j], we can shift P by six chars and compare T[j] with P[1] without missing any possible matches Number of comparisons saved: 7 abcxabc T abcxabcde P mismatch abcxabc T abcxabcde Naïve approach: abcxabcde Should not be a c abcxabcde ?

12. KMP algorithm: pre-processing Key: the reasoning is done without even knowing what string T is. Only the location of mismatch in P must be known. t t’ P t x T y t P y z z Pre-processing: for any position i in P, find P[1..i]’s longest proper suffix, t = P[j..i], such that t matches to a prefix of P, t’, and the next char of t is different from the next char of t’ (i.e., y ≠ z) For each i, let sp(i) = length(t) ij ij

13. KMP algorithm: shift rule t t’ P t x T y t P y z z Shift rule: when a mismatch occurred between P[i+1] and T[k], shift P to the right by i – sp(i) chars and compare x with z. This shift rule can be implicitly represented by creating a failure link between y and z. Meaning: when a mismatch occurred between x on T and P[i+1], resume comparison between x and P[sp(i)+1]. ij ijsp(i)1

14. Failure Link Example P: aataac aataac sp(i) 010020 aaataaat aat aac If a char in T fails to match at pos 6, re-compare it with the char at pos 3 (= 2 + 1)

15. Another example P: abababc abababc Sp(i) 0000040 ababa ababc If a char in T fails to match at pos 7, re-compare it with the char at pos 5 (= 4 + 1) abababab abab

16. KMP Example using Failure Link aataac aataac ^^* T: aacaataaaaataaccttacta aataac.* aataac ^^^^^* aataac..* aataac.^^^^^ Time complexity analysis: Each char in T may be compared up to n times. A lousy analysis gives O(mn) time. More careful analysis: number of comparisons can be broken to two phases: Comparison phase: the first time a char in T is compared to P. Total is exactly m. Shift phase. First comparisons made after a shift. Total is at most m. Time complexity: O(2m) Implicit comparison

17. KMP algorithm using DFA (Deterministic Finite Automata) P: aataac 123450 aa taac 6 a t If the next char in T is t after matching 5 chars, go to state 3 aataac If a char in T fails to match at pos 6, re-compare it with the char at pos 3 a Failure link DFA a All other inputs goes to state 0.

18. DFA Example T: aacaataataataaccttacta Each char in T will be examined exactly once. Therefore, exactly m comparisons are made. But it takes longer to do pre-processing, and needs more space to store the FSA. 1201234534534560001001 123450 aa taac 6 a t a DFA a

19. Difference between Failure Link and DFA Failure link –Preprocessing time and space are O(n), regardless of alphabet size –Comparison time is at most 2m (at least m) DFA –Preprocessing time and space are O(n |  |) May be a problem for very large alphabet size For example, each “char” is a big integer Chinese characters –Comparison time is always m.

20. The set matching problem Find all occurrences of a set of patterns in T First idea: run KMP or BM for each P –O(km + n) k: number of patterns m: length of text n: total length of patterns Better idea: combine all patterns together and search in one run

21. A simpler problem: spell-checking A dictionary contains five words: –potato –poetry –pottery –science –school Given a document, check if any word is (not) in the dictionary –Words in document are separated by special chars. –Relatively easy.

22. Keyword tree for spell checking O(n) time to construct. n: total length of patterns. Search time: O(m). m: length of text Common prefix only need to be compared once. What if there is no space between words? p o t a t o e t r y t e r y s c i e n c e hoo l 1 2 3 4 5 This version of the potato gun was inspired by the Weird Science team out of Illinois

23. Aho-Corasick algorithm Basis of the fgrep algorithm Generalizing KMP –Using failure links Example: given the following 4 patterns: –potato –tattoo –theater –other

24. Keyword tree p o t a t o t e r 0 t h e r 1 2 3 4 a t t o o h a t e

25. p o t a t o t e r 0 t h e r 1 2 3 4 a t t o o h a t e potherotathxythopotattooattoo

26. Keyword tree p o t a t o t e r 0 t h e r 1 2 3 4 a t t o o h a t e O(mn) m: length of text. n: length of longest pattern potherotathxythopotattooattoo

27. Keyword Tree with a failure link p o t a t o t e r 0 t h e r 1 2 3 4 a t t o o h a t e potherotathxythopotattooattoo

28. Keyword Tree with a failure link p o t a t o t e r 0 t h e r 1 2 3 4 a t t o o h a t e potherotathxythopotattooattoo

29. Keyword Tree with all failure links p o t a t o t e r 0 t h e r 1 2 3 4 a t t o o h a t e

30. Example p o t a t o t e r 0 t h e r 1 2 3 4 a t t o o h a t e potherotathxythopotattooattoo

31. Example p o t a t o t e r 0 t h e r 1 2 3 4 a t t o o h a t e potherotathxythopotattooattoo

32. Example p o t a t o t e r 0 t h e r 1 2 3 4 a t t o o h a t e potherotathxythopotattooattoo

33. Example p o t a t o t e r 0 t h e r 1 2 3 4 a t t o o h a t e potherotathxythopotattooattoo

34. Example p o t a t o t e r 0 t h e r 1 2 3 4 a t t o o h a t e potherotathxythopotattooattoo

35. Aho-Corasick algorithm O(n) preprocessing, and O(m+k) searching. –n: total length of patterns. –m: length of text –k is # of occurrence.

36. Suffix Tree All algorithms we talked about so far preprocess pattern(s) –Boyer-Moore: fastest in practice. O(m) worst case. –KMP: O(m) –Aho-Corasick: O(m) In some cases we may prefer to pre-process T –Fixed T, varying P Suffix tree: basically a keyword tree of all suffixes

37. Suffix tree T: xabxac Suffixes: 1.xabxac 2.abxac 3.bxac 4.xac 5.ac 6.c a b x a c b x a c c c x a b x a c c 1 2 3 4 5 6 Naïve construction: O(m 2 ) using Aho-Corasick. Smarter: O(m). Very technical. big constant factor Difference from a keyword tree: create an internal node only when there is a branch

38. Suffix tree implementation Explicitly labeling sequence end T: xabxa$ a b x a b x a x a b x a 1 2 3 a b x a b x a x a b x a 1 2 3 $ $ $ $ $ 4 5 One-to-one correspondence of leaves and suffixes |T| leaves, hence < |T| internal nodes

39. Suffix tree implementation Implicitly labeling edges T: xabxa$ a b x a b x a x a b x a 1 2 3 $ $ $ $ $ 4 5 2:2 3:$ 1 2 3 $ $ 4 5 1:2 3:$ |Tree(T)| = O(|T| + size(edge labels))

40. Suffix links Similar to failure link in a keyword tree Only link internal nodes having branches x a b c d e f g h i j a b c d e f g h i j P: xabcf f

41. ST Application 1: pattern matching Find all occurrence of P=xa in T –Find node v in the ST that matches to P –Traverse the subtree rooted at v to get the locations a b x a c b x a c c c x a b x a c c 1 2 3 4 5 6 T: xabxac O(m) to construct ST (large constant factor) O(n) to find v – linear to length of P instead of T! O(k) to get all leaves, k is the number of occurrence. Asymptotic time is the same as KMP. ST wins if T is fixed. KMP wins otherwise.

42. ST application 2: repeats finding Genome contains many repeated DNA sequences Repeat sequence length: Varies from 1 nucleotide to millions –Genes may have multiple copies (50 to 10,000) –Highly repetitive DNA in some non-coding regions 6 to 10bp x 100,000 to 1,000,000 times Problem: find all repeats that are at least k- residues long and appear at least p times in the genome

43. Repeats finding at least k-residues long and appear at least p times in the seq –Phase 1: top-down, count label lengths (L) from root to each node –Phase 2: bottom-up: count # of leaves descended from each internal node (L, N) For each node with L >= k, and N >= p, print all leaves O(m) to traverse tree

44. Maximal repeats finding 1.Right-maximal repeat –S[i+1..i+k] = S[j+1..j+k], –but S[i+k+1] != S[j+k+1] 2.Left-maximal repeat –S[i+1..i+k] = S[j+1..j+k] –But S[i] != S[j] 3.Maximal repeat –S[i+1..i+k] = S[j+1..j+k] –But S[i] != S[j], and S[i+k+1] != S[j+k+1] acatgacatt 1.cat 2.aca 3.acat

45. Maximal repeats finding Find repeats with at least 3 bases and 2 occurrence –right-maximal: cat –Maximal: acat –left-maximal: aca 5:e 2 4 1234567890 acatgacatt 5:e 5 c a t t 7 c a t t 6 a 3 1 t 8 t t t 9 10 $

46. Maximal repeats finding How to find maximal repeat? –A right-maximal repeats with different left chars 5:e 2 4 1234567890 acatgacatt 5:e 5 c a t t 7 c a t t 6 a 3 1 t 8 t t t 9 10 $ Left char = [] gcc aa

47. Joint Suffix Tree (JST) Build a ST for more than two strings Two strings S 1 and S 2 S* = S 1 & S 2 Build a suffix tree for S* in time O(|S 1 | + |S 2 |) The separator will only appear in the edge ending in a leaf (why?)

48. Joint suffix tree example S1 = abcd S2 = abca S* = abcd&abca$ a b c d & a b c a bcd&abcabcd&abca c d & a b c d d & a b c d & a b c d a a a $ 1,1 2,1 1,2 1,3 1,4 2,2 2,3 2,4 (2, 0) useless Seq ID Suffix ID

49. To Simplify We don’t really need to do anything, since all edge labels were implicit. The right hand side is more convenient to look at a b c d & a b c a bcd&abcabcd&abca c d & a b c d d & a b c d & a b c d a a a $ 1,1 2,1 1,2 1,3 1,4 2,2 2,3 2,4 useless a b c d bcdbcd c d d a a a $ 1,1 2,1 1,2 1,3 1,4 2,2 2,3 2,4

50. Application 1 of JST Longest common substring between two sequences Using smith-waterman –Gap = mismatch = -infinity. –Quadratic time Using JST –Linear time –For each internal node v, keep a bit vector B –B[1] = 1 if a child of v is a suffix of S1 –Bottom-up: find all internal nodes with B[1] = B[2] = 1 (green nodes) –Report a green node with the longest label –Can be extended to k sequences. Just use a bit vector of size k. a b c d bcdbcd c d d a a a $ 1,1 2,1 1,2 1,3 1,4 2,2 2,3 2,4

51. Application 2 of JST Substring problem for sequence databases –Given: A fixed database of sequences (e.g., individual genomes) –Given: A short pattern (e.g., DNA signature) –Q: Does this DNA signature belong to any individual in the database? i.e. the pattern is a substring of some sequences in the database Aho-Corasick doesn’t work –This can also be used to design signatures for individuals Build a JST for the database seqs Match P to the JST Find seq IDs from descendents a b c d bcdbcd c d d a a a $ 1,1 2,1 1,2 1,3 1,4 2,2 2,3 2,4 Seqs: abcd, abca P1: cd P2: bc

52. Application 3 of JST Given K strings, find all k-mers that appear in at least (or at most) d strings Exact motif finding problem L< k L >= k B = BitOR(1010, 0011) = 1011 cardinal(B) = 3 3,x 4,x B = 0011 1,x B = 1010 cardinal(B) >= 3

53. Combinatorial motif finding Idea 1: find all k-mers that appeared at least m times –m may be chosen such that # occurrence is statistically significant –Problem: most motifs have divergence. Each variation may only appear once. Idea 2: find all k-mers, considering IUPAC nucleic acid codes –e.g. ASGTKTKAC, S = C/G, K = G/T –Still inflexible Idea 3: find k-mers that approximately appeared at least m times –i.e. allow some mismatches

54. Combinatorial motif finding Given a set of sequences S = {x 1, …, x n } A motif W is a consensus string w 1 …w K Find motif W * with “best” match to x 1, …, x n Definition of “best”: d(W, x i ) = min hamming dist. between W and a word in x i d(W, S) =  i d(W, x i ) W* = argmin( d(W, S) )

55. Exhaustive searches 1. Pattern-driven algorithm: For W = AA…A to TT…T (4 K possibilities) Find d( W, S ) Report W* = argmin( d(W, S) ) Running time: O( K N 4 K ) (where N =  i |x i |) Guaranteed to find the optimal solution.

56. Exhaustive searches 2. Sample-driven algorithm: For W = a K-char word in some x i Find d( W, S ) Report W* = argmin( d( W, S ) ) OR Report a local improvement of W * Running time: O( K N 2 )

57. Exhaustive searches Problem with sample-driven approach: If: –True motif does not occur in data, and –True motif is “weak” Then, –random strings may score better than any instance of true motif

58. Example E. coli. Promoter “TATA-Box” ~ 10bp upstream of transcription start TACGAT TAAAAT TATACT GATAAT TATGAT TATGTT Consensus: TATAAT Each instance differs at most 2 bases from the consensus None of the instances matches the consensus perfectly

59. Heuristic methods Cannot afford exhaustive search on all patterns Sample-driven approaches may miss real patterns However, a real pattern should not differ too much from its instances in S Start from the space of all words in S, extend to the space with real patterns

60. Extended sample-driven (ESD) approaches Hybrid between pattern-driven and sample-driven Assume each instance does not differ by more than α bases to the motif (  usually depends on k) motif instance  The real motif will reside in the  - neighborhood of some words in S. Instead of searching all 4 K patterns, we can search the  -neighborhood of every word in S. α-neighborhood

61. Extended sample-driven (ESD) approaches Naïve: N K α 3 α NK # of patterns to test# of words in sequences

62. Better idea Using a joint suffix tree, find all patterns that: –Have length K –Appeared in at least m sequences with at most α mismatches Post-processing

63. WEEDER: algorithm sketch A list containing all eligible nodes: with at most α mismatches to P For each node, remember #mismatches accumulated (e  α ), and a bit vector (B) for seq occ, e.g. [011100010] Bit OR all B’s to get seq occurrence for P Suppose #occ >= m –Pattern still valid Now add a letter ACGTTACGTT Current pattern P, |P| < K (e, B) # mismatches Seq occ

64. WEEDER: algorithm sketch Simple extension: no branches. –No change to B –e may increase by 1 or no change –Drop node if e > α Branches: replace a node with its child nodes –Drop if e > α –B may change Re-do Bit OR using all B’s Try a different char if #occ < m Report P when |P| = K ACGTTAACGTTA Current pattern P (e, B)

65. WEEDER: complexity Can get all patterns in time O(Nn(K choose α) 3 α ) ~ O(N nK α 3 α ). n: # sequences. Needed for Bit OR. Better than O(KN 4 K ) and O(N K α 3 α NK) since usually α << K K α 3 α may still be expensive for large K –E.g. K = 20, α = 6

66. WEEDER: More tricks Eligible nodes: with at most α mismatches to P Eligible nodes: with at most min(  L, α) mismatches to P –L: current pattern length –  : error ratio –Require that mismatches to be somewhat evenly distributed among positions Prune tree at length K ACGTTAACGTTA Current pattern P

67. Suffix Tree Memory Footprint The space requirements of suffix trees can become prohibitive –|Tree(T)| is about 20|T| in practice Suffix arrays provide one solution.

68. Suffix Arrays Very space efficient (m integers) Pattern lookup is nearly O(n) in practice –O(n + log 2 m) worst case with 2m additional integers –Independent of alphabet size! Easiest to describe (and construct) using suffix trees –Other (slower) methods exist a b x a b x a x a b x a 1 5 3 $ $ $ $ $ 4 2 52341 abxa$ a$ bxa$xa$xabxa$ 1. xabxa 2. abxa 3. bxa 4. xa 5. a

69. Suffix array construction Build suffix tree for T$ Perform “lexical” depth-first search of suffix tree –output the suffix label of each leaf encountered Therefore suffix array can be constructed in O(m) time.

70. Suffix array pattern search If P is in T, then all the locations of P are consecutive suffixes in Pos. Do binary search in Pos to find P! –Compare P with suffix Pos(m/2) –If lexicographically less, P is in first half of T –If lexicographically more, P is in second half of T –Iterate!

71. Suffix array pattern search T: xabxa$ P: abx a b x a b x a x a b x a 1 5 3 $ $ $ $ $ 4 2 52341 abxa$ a$ bxa$xa$xabxa$ L RM R M

72. Suffix array binary search How long to compare P with suffix of T? –O(n) worst case! Binary search on Pos takes O(n log m) time Worst case will be rare –occur if many long prefixes of P appear in T In random or large alphabet strings –expect to do less than log m comparisons O(n + log m) running time when combined with LCP table –suffix tree = suffix array + LCP table

73. Summary One T, one P –Boyer-Moore is the choice –KMP works but not the best One T, many P –Aho-Corasick –Suffix Tree (array) One fixed T, many varying P –Suffix tree (array) Two or more T’s –Suffix tree, joint suffix tree Alphabet independent Alphabet dependent