1. CS 5263 & CS 4233 Bioinformatics
Motif finding

2. What is a (biological) motif?
A motif is a recurring fragment, theme or pattern
Sequence motif: a sequence pattern of nucleotides in a DNA sequence or amino acids in a protein
Structural motif: a pattern in a protein structure formed by the spatial arrangement of amino acids.
Network motif: patterns that occur in different parts of a network at frequencies much higher than those found in randomized network
Commonality:
higher frequency than would be expected by chance
Has, or is conjectured to have, a biological significance

3. Sequence motif finding
Given: a set of sequences
Goal: find sequence motifs that appear in all or the majority of the sequences, and are likely associated with some functions
In DNA: regulatory sequences
Other names: transcription factor binding sites, transcription factor binding motifs, cis-regulatory elements, cis-regulatory motifs, DNA motifs, etc.
In protein: functional/structural domains

4. Roadmap Biological background Representation of motifs
Algorithms for finding motifs
Other issues
Search for instances of given motifs
Distinguish functional vs non-functional motifs

5. Biological background for motif finding

6. Genome is fixed – Cells are dynamic
A genome is static
(almost) Every cell in our body has a copy of the same genome
A cell is dynamic
Responds to internal/external conditions
Most cells follow a cell cycle of division
Cells differentiate during development

7. Gene regulation … is responsible for the dynamic cell
Gene expression (production of protein) varies according to:
Cell type
Cell cycle
External conditions
Location
Etc.

8. Where gene regulation takes place
Opening of chromatin
Transcription
Translation
Protein stability
Protein modifications

9. Transcriptional Regulation of genes
Transcription Factor (TF)
(Protein)
RNA polymerase
(Protein)
DNA
Promoter
Gene

10. Transcriptional Regulation of genes
Transcription Factor (TF)
(Protein)
RNA polymerase
(Protein)
DNA
Gene
TF binding site, cis-regulatory element

11. Transcriptional Regulation of genes
Transcription Factor
(Protein)
RNA polymerase
DNA
Gene
TF binding site, cis-regulatory element

12. Transcriptional Regulation of genes
New protein
RNA polymerase
Transcription Factor
DNA
Gene
TF binding site, cis-regulatory element

13. The Cell as a Regulatory Network
If C then D
gene D A B C
Make D
If B then NOT D D
If A and B then D
gene B D C
Make B
If D then B

14. Transcription Factors Binding to DNA
Transcriptional regulation:
Transcription factors bind to DNA
Binding recognizes specific DNA substrings:
Regulatory motifs

15. Experimental methods DNase footprinting
Tedious
Time-consuming
High-throughput techniques: ChIP-chip, ChIP-seq
Expensive
Other limitations

16. Computational methods for finding cis-regulatory motifs.
Given a collection of genes that are believed to be regulated by the same/similar protein
Co-expressed genes
Evolutionarily conserved genes
Find the common TF-binding motif from promoters

17. Essentially a Multiple Local Alignment.
instance
Find “best” multiple local alignment
Multidimensional Dynamic Programming?
Heuristics must be used

18. Characteristics of cis-Regulatory Motifs
Tiny (6-12bp)
Intergenic regions are very long
Highly Variable
~Constant Size
Because a constant-size transcription factor binds
Often repeated
Often conserved

19. Motif representation Collection of exact words
{ACGTTAC, ACGCTAC, AGGTGAC, …}
Consensus sequence (with wild cards)
{AcGTgTtAC}
{ASGTKTKAC} S=C/G, K=G/T (IUPAC code)
Position-specific weight matrices (PWM)

20. Position-Specific Weight Matrix1 2 3 4 5 6 7 8 9 A
.97
.10
.02
.03
.01
.05
.85 C
.40
.04 G
.95 .3 T
.90
.45 .6
.91
A S G T K T K A C

21. Sequence Logo frequency 1 2 3 4 5 6 7 8 9 A .97 .10 .02 .03 .01 .05
.85 C
.40
.04 G
.95 .3 T
.90
.45 .6
.91

22. Sequence Logo 1 2 3 4 5 6 7 8 9 A
.97
.10
.02
.03
.01
.05
.85 C
.40
.04 G
.95 .3 T
.90
.45 .6
.91

23. Entropy and information content
Entropy: a measure of uncertainty
The entropy of a random variable X that can assume the n different values x1, x2, , xn with the respective probabilities p1, p2, , pn is defined as

24. Entropy and information content
Example: A,C,G,T with equal probability
H = 4 * (-0.25 log2 0.25) = log2 4 = 2 bits
Need 2 bits to encode (e.g. 00 = A, 01 = C, 10 = G, 11 = T)
Maximum uncertainty
50% A and 50% C:
H = 2 * (-0. 5 log2 0.5) = log2 2 = 1 bit
100% A
H = 1 * (-1 log2 1) = 0 bit
Minimum uncertainty
Information: the opposite of uncertainty
I = maximum uncertainty – entropy
The above examples provide 0, 1, and 2 bits of information, respectively

25. Entropy and information content1 2 3 4 5 6 7 8 9 A
.97
.10
.02
.03
.01
.05
.85 C
.40
.04 G
.95 .3 T
.90
.45 .6
.91 H
.24
1.72
.36
.63
1.60
0.24
1.40
0.85
0.58 I
1.76
0.28
1.64
1.37
0.40
0.60
1.15
1.42
Mean
Total
10.4
Expected occurrence in random DNA: 1 / = 1 / 1340
Expected occurrence of an exact 5-mer: 1 / 210 = 1 / 1024

26. Sequence Logo 1 2 3 4 5 6 7 8 9 A
.97
.10
.02
.03
.01
.05
.85 C
.40
.04 G
.95 .3 T
.90
.45 .6
.91 I
1.76
0.28
1.64
1.37
0.40
0.60
1.15
1.42

27. Real example E. coli. Promoter
“TATA-Box” ~ 10bp upstream of transcription start
TACGAT
TAAAAT
TATACT
GATAAT
TATGAT
TATGTT
Consensus: TATAAT
Note: none of the instances matches the consensus perfectly

28. Finding Motifs

29. Classification of approaches
Combinatorial algorithms
Based on enumeration of words and computing word similarities
Probabilistic algorithms
Construct probabilistic models to distinguish motifs vs non-motifs

30. Combinatorial motif finding
Given a set of sequences S = {x1, …, xn}
A motif W is a consensus string w1…wK
Find motif W* with “best” match to x1, …, xn
Definition of “best”:
d(W, xi) = min hamming dist. between W and a word in xi
d(W, S) = i d(W, xi)
W* = argmin( d(W, S) )

31. Exhaustive searches 1. Pattern-driven algorithm:
For W = AA…A to TT…T (4K possibilities)
Find d( W, S )
Report W* = argmin( d(W, S) )
Running time: O( K N 4K )
(where N = i |xi|)
Guaranteed to find the optimal solution.

32. Exhaustive searches 2. Sample-driven algorithm:
For W = a K-char word in some xi
Find d( W, S )
Report W* = argmin( d( W, S ) )
OR Report a local improvement of W*
Running time: O( K N2 )

33. Exhaustive searches Problem with sample-driven approach: If: Then,
True motif does not occur in data, and
True motif is “weak”
Then,
random strings may score better than any instance of true motif

34. Example E. coli. Promoter
“TATA-Box” ~ 10bp upstream of transcription start
TACGAT
TAAAAT
TATACT
GATAAT
TATGAT
TATGTT
Consensus: TATAAT
Each instance differs at most 2 bases from the consensus
None of the instances matches the consensus perfectly

35. Heuristic methods Cannot afford exhaustive search on all patterns
Sample-driven approaches may miss real patterns
However, a real pattern should not differ too much from its instances in S
Start from the space of all words in S, extend to the space with real patterns

36. Some of the popular tools
Consensus (Hertz & Stormo, 1999)
WINNOWER (Pevzner & Sze, 2000)
MULTIPROFILER (Keich & Pevzner, 2002)
PROJECTION (Buhler & Tompa, 2001)
WEEDER (Pavesi et. al. 2001)
And dozens of others

37. Extended sample-driven (ESD) approaches
Hybrid between pattern-driven and sample-driven
Assume each instance does not differ by more than α bases to the motif ( usually depends on k)
motif
instance 
The real motif will reside in the -neighborhood of some words in S.
Instead of searching all 4K patterns, we can search the -neighborhood of every word in S.
α-neighborhood

38. Extended sample-driven (ESD) approaches
Naïve: N Kα 3α NK
# of patterns to test
# of words in sequences

39. Better idea Using a joint suffix tree, find all patterns that:
Have length K
Appeared in at least m sequences with at most α mismatches
Post-processing
Details later

40. Probabilistic modeling approaches
for motif finding

41. Probabilistic modeling approaches
A motif model
Usually a PWM
M = (Pij), i = 1..4, j = 1..k, k: motif length
A background model
Usually the distribution of base frequencies in the genome (or other selected subsets of sequences)
B = (bi), i = 1..4
A word can be generated by M or B

42. Expectation-Maximization
For any word W,
P(W | M) = PW[1] 1 PW[2] 2…PW[K] K
P(W | B) = bW[1] bW[2] …bW[K]
Let  = P(M), i.e., the probability for any word to be generated by M.
Then P(B) = 1 - 
Can compute the posterior probability P(M|W) and P(B|W)
P(M|W) ~ P(W|M) * 
P(B|W) ~ P(W|B) * (1-)

43. Expectation-Maximization
Initialize:
Randomly assign each word to M or B
Let Zxy = 1 if position y in sequence x is a motif, and 0 otherwise
Estimate parameters M, , B
Iterate until converge:
E-step: Zxy = P(M | X[y..y+k-1]) for all x and y
M-step: re-estimate M,  given Z (B usually fixed)

44. Expectation-Maximization
position 1 1
Initialize
E-step 5
probability 5 9 9
M-step
E-step: Zxy = P(M | X[y..y+k-1]) for all x and y
M-step: re-estimate M,  given Z

45. MEME Multiple EM for Motif Elicitation Bailey and Elkan, UCSD
Multiple starting points
Multiple modes: ZOOPS, OOPS, TCM

46. Gibbs Sampling
Another very useful technique for estimating missing parameters
EM is deterministic
Often trapped by local optima
Gibbs sampling: stochastic behavior to avoid local optima

47. Gibbs Sampling Initialize: Randomly assign each word to M or B
Let Zxy = 1 if position y in sequence x is a motif, and 0 otherwise
Estimate parameters M, B, 
Iterate:
Randomly remove a sequence X* from S
Recalculate model parameters using S \ X*
Compute Zx*y for X*
Sample a y* from Zx*y.
Let Zx*y = 1 for y = y* and 0 otherwise

48. Gibbs Sampling
position
probability
Sampling
Gibbs sampling: sample one position according to probability
Update prediction of one training sequence at a time
Viterbi: always take the highest
EM: take weighted average
Simultaneously update predictions of all sequences

49. Gibbs sampling motif finders
Gibbs Sampler
First appeared as: Larence et.al. Science 262(5131):
Continually developed and updated. webpage
The newest version: Thompson et. al. Nucleic Acids Res. 35 (s2):W232-W237
AlignACE
Hughes et al., J. of Mol Bio, ;296(5):
Allow don’t care positions
Additional tools to scan motifs on new seqs, and to compare and group motifs
BioProspector, X. Liu et. al. PSB 2001 , an improvement of AlignACE
Liu, Brutlag and Liu. Pac Symp Biocomput. 2001;:
Allow two-block motifs
Consider higher-order markov models

50. Limits of Motif Finders
???
gene
Given upstream regions of coregulated genes:
Increasing length makes motif finding harder – random motifs clutter the true ones
Decreasing length makes motif finding harder – true motif missing in some sequences

51. Challenging problem (k, d)-motif challenge problem
d mutations
n = 20 k
L = 600
(k, d)-motif challenge problem
Many algorithms fail at (15, 4)-motif for n = 20 and L = 600
Combinatorial algorithms usually work better on challenge problem
However, they are usually designed to find (k, d)-motifs
Performance in real data varies

52. (15, 4)-motif Information content: 11.7 bits
~ 6mers. Expected occurrence 1 per 3k bp

53. Actual Results by MEME llr = 163 E-value = 3.2e+005

54. Motif finding in practice
Where the input come from?
Possibility 1: transcriptomic studies
E.g. microarray, RNA-seq (later)
Possibility 2: phylogenetic analysis (not covered)
Possibility 3: ChIP-chip
Possibility 4: ChIP-seq

55. Chromatin Immunoprecipitation (ChIP)
ChIP is a method to investigate protein-DNA interaction in vivo.
The output of ChIP is enriched fragments of DNA that were bound by a particular protein.
The identity of DNA fragments need to be further determined by a second method.

56. ChIPSeq Workflow
ChIP
Size Selection
Sequencing
Mapping onto Genome

57. Array of intergenic sequences from the whole genome
ChIP-chip
ChIP
Array of intergenic sequences from the whole genome

58. How to make sense of the motifs?
Each program usually reports a number of motifs (tens to hundreds)
Many motifs are variations of each other
Each program also report some different ones
Each program has its own way of scoring motifs
Best scored motifs often not interesting
AAAAAAAA
ACACACAC
TATATATAT

59. How to make sense of the motifs?
Now we’ve found some pretty-looking motifs
This is probably the easiest step
What to do next?
Are they real?
How do we find more instances in the rest of the genome?
What are their functional meaning?
Motifs => regulatory networks

60. How to make sense of the motifs?
Combine results from different algorithms usually helpful
Ones that appeared multiple times are probably more interesting
Except simple repeats like AAAAA or ATATATATA
Cluster motifs into groups.
Compare with known motifs in database
TRANSFAC
JASPAR
YPD (yeast promoter database)

61. Strategies to improve results
How to tell real motifs (functional) from noises? Statistical test of significance.
Enrichment in target sequences vs background sequences
Background set B
Target set T
Assumed to contain a common motif, P
Assumed to not contain P, or with very low frequency
Ideal case: every sequence in T has P, no sequence in B has P

62. Statistical test for significance
Background set + target set
B + T P
Target set T M N
P appeared in n sequences
P appeared in m sequences
If n / N >> m / M
P is enriched (over-represented) in T
Statistical significance?
If we randomly draw N sequences from (B+T), how likely we will see at least n sequences having P?

63. Hypergeometric distribution
A box with M balls (seqs), of which m are red (with motifs), and the rest are blue (without motifs).
Red ball: sequences with motifs
Blue ball: sequences without motifs
We randomly draw N balls (seqs) from the box
What’s the probability we’ll see n red balls?
# of choices to have n red balls
Total # of choices to draw N balls

64. Cumulative hypergeometric test for motif significance
We are interested in: if we randomly pick m balls, how likely that we’ll see at least n red balls?
Null hypothesis: our selection is random.
Alternative hypothesis: our selection favored red balls.
When prob is small, we reject the null hypothesis.
Equivalent: we accept the alternative hypothesis
(The number of red balls is larger than expected).

65. Example Yeast genome has 6000 genes
Select 50 genes believed to be co-regulated by a common TF
Found a motif from the promoter seqs of these 50 genes
The motif appears in 20 of these 50 genes
In the rest of the genome, 100 genes have this motif
M = 6000, N = 50, m = = 120, n = 20
Intuitively:
m/M = 120/6000=1/50. (1 out 50 genes has the motif)
N = 50, would expect only 1 gene in the target set to have the motif
20-fold enrichment
P-value = cHyperGeom(20; 6000, 50, 120) = 6 x 10-22
This motif is significantly enriched in the set of genes

66. ROC curve for motif significance
Motif is usually a PWM
Any word will have a score
Typical scoring function: Log (P(W | M) / P(W | B))
W: a word.
M: a PWM.
B: background model
To determine whether motif M occurred in a sequence, a cutoff has to be decided
Different cutoffs give different # of occurrences
Stringent cutoff: low occurrence in both + and - sequences
Loose cutoff: high occurrence in both + and - sequences
It may be better to look at a range of cutoffs

67. ROC curve for motif significance
Background set + target set
B + T P
Target set T M N
Given a score cutoff
Appeared in n sequences
Appeared in m sequences
With different score cutoff, will have different m and n
Assume you want to use P to classify T and B
Sensitivity: n / N
Specificity: (M-N-m+n) / (M-N)
False Positive Rate = 1 – specificity: (m – n) / (M-N)
With decreasing cutoff, sensitivity , FPR 

68. ROC curve for motif significance
A good cutoff 1
Lowest cutoff. Every sequence has the motif. Sensitivity = 1. specificity = 0.
ROC-AUC: area under curve.
1: the best. 0.5: random.
Motif 1 is more enriched than motif 2.
sensitivity
Motif 1
Motif 2
Random
1-specificity 1
Highest cutoff. No motif can pass the cutoff. Sensitivity = 0. specificity = 1.

69. Other strategies Cross-validation
Randomly divide sequences into 10 sets, hold 1 set for test.
Do motif finding on 9 sets. Does the motif also appear in the testing set?
Phylogenetic conservation information
Does a motif also appears in the homologous genes of another species?
Strongest evidence
However, will not be able to find species-specific ones

70. Other strategies Finding motif modules Location preference
Will two motifs always appear in the same gene?
Location preference
Some motifs appear to be in certain location
E.g., within bp upstream to transcription start
If a detected motif has strong positional bias, may be a sign of its function
Evidence from other types of data sources
Do the genes having the motif always have similar activities (gene expression levels) across different conditions?
Interact with the same set of proteins?
Similar functions?
etc.

71. To search for new instances
Usually many false positives
Score cutoff is critical
Can estimate a score cutoff from the “true” binding sites
Motif finding
Scoring function
Log (P(W | M) / P(W | B))
A set of scores for the “true” sites. Take mean - std as a cutoff.
(or a cutoff such that the majority of “true” sites can be predicted).

72. To search for new instances
Use other information, such as positional biases of motifs to restrict the regions that a motif may appear
Use gene expression data to help: the genes having the true motif should have similar activities
Risk of circular reasoning: most likely this is how you get the initial sequences to do motif finding
Phylogenetic conservation is the key

73. References
D’haeseleer P (2006) What are DNA sequence motifs? NATURE BIOTECHNOLOGY, 24 (4):
D’haeseleer P (2006) How does DNA sequence motif discovery work? NATURE BIOTECHNOLOGY, 24 (8):
MacIsaac KD, Fraenkel E (2006) Practical strategies for discovering regulatory DNA sequence motifs. PLoS Comput Biol 2(4): e36
Lawrence CE et. al. (1993) Detecting Subtle Sequence Signals: A Gibbs Sampling Strategy for Multiple Alignment, Science, 262(5131):