1. Sequence motifs, information content, logos, and HMM’s
Morten Nielsen,
CBS, BioCentrum,
DTU

2. Outline Pattern recognition Multiple alignment and sequence motifs
Regular expression and probabilities
Multiple alignment and sequence motifs
Weight matrix construction and consensus sequence
Sequence weighting
Low (pseudo) counts
Information content
Sequence logos
Mutual information
Example from the real world
HMM’s and profile HMM’s
TMHMM (trans-membrane protein)
Gene finding
Links to HMM packages

3. Pattern recognition ALAKAAAAM ALAKAAAAV GMNERPILV GILGFVFTM TLNAWVKVV
10 peptides from MHCpep database
Bind to the MHC complex A*0201
Which of the are most likely to bind?
FLLTRILTI
WLDQVPFSV
TVILGVLLL
Regular expression
X1[LMIV]2X3…X8[MVL]9
2 and 3 will bind and 1 will not bind
Cannot tell if 2 if more likely to bind
Truth is that 1 and 2 binds and 1 binds the strongest. 3 does not bind
A probabilistic model can capture this!
ALAKAAAAM
ALAKAAAAV
GMNERPILV
GILGFVFTM
TLNAWVKVV
KLNEPVLLL
AVVPFIVSV

4. Multiple alignment and sequence motifs
Core
Consensus sequence
Weight matrices
Problems
Sequence weights
Low counts
MLEFVVEADLPGIKA
MLEFVVEFALPGIKA
MLEFVVEFDLPGIAA
YLQDSDPDSFQD
---GSDTITLPCRMKQFINMWQE
---RNQEERLLADLMQNYDPNLR
YDPNLRPAERDSDVVNVSLK------
NVSLKLTLTNLISLNEREEA---
----EREEALTTNVWIEMQWCDYR
WCDYRLRWDPRDYEGLWVLR---
--LWVLRVPSTMVWRPDIVLEN
IVLENNVDGVFEVALYCNVL-
YCNVLVSPDGCIYWLPPAIF
PPAIFRSACSISVTYFPFDW----
*********
FVVEFDLPG
Consensus

5. Sequences weighting 1 - Clustering (slow, but accurate)
MLEFVVEADLPGIKA
MLEFVVEFALPGIKA
MLEFVVEFDLPGIAA
YLQDSDPDSFQD
---GSDTITLPCRMKQFINMWQE
---RNQEERLLADLMQNYDPNLR
YDPNLRPAERDSDVVNVSLK------
NVSLKLTLTNLISLNEREEA---
----EREEALTTNVWIEMQWCDYR
WCDYRLRWDPRDYEGLWVLR---
--LWVLRVPSTMVWRPDIVLEN
IVLENNVDGVFEVALYCNVL-
YCNVLVSPDGCIYWLPPAIF
PPAIFRSACSISVTYFPFDW----
********* }
Homologous sequences
Weight = 1/n (1/3)
Consensus sequence
YRQELDPLV
Previous
FVVEFDLPG

6. Sequences weighting 2 - Henikoff & Henikoff (fast)
FVVEADLPG 0.37
FVVEFALPG 0.43
FVVEFDLPG 0.32
YLQDSDPDS 0.59
MKQFINMWQ 0.90
LMQNYDPNL 0.68
PAERDSDVV 0.75
LKLTLTNLI 0.85
VWIEMQWCD 0.84
YRLRWDPRD 0.51
WRPDIVLEN 0.71
VLENNVDGV 0.59
YCNVLVSPD 0.71
FRSACSISV 0.75
waa’ = 1/rs
r: Number of different aa in a column
s: Number occurrences
Normalize S waa= 1 for each column
Sequence weight is sum of waa in sequence
F: r=7 (FYMLPVW), s=4
w’=1/28, w = 0.055
Y: s=3, w`=1/21, w = 0.073
M,P,W: s=1, w’=1/7, w = 0.218
L,V: s=2, w’=1/14, w = 0.109

7. Low count correction Limited number of dataP1
Limited number of data
Poor sampling of sequence space
I is not found at position P1. Does this mean that I can never be found at P1?
No! Use Blosum matrix to estimate pseudo frequency of I
MLEFVVEADLPGIKA
MLEFVVEFALPGIKA
MLEFVVEFDLPGIAA
YLQDSDPDSFQD
-GSDTITLPCRMKQFINMWQE
-RNQEERLLADLMQNYDPNLR
-----YDPNLRPAERDSDVVNVSLK------
NVSLKLTLTNLISLNEREEA---
--EREEALTTNVWIEMQWCDYR
WCDYRLRWDPRDYEGLWVLR---
LWVLRVPSTMVWRPDIVLEN
IVLENNVDGVFEVALYCNVL-
YCNVLVSPDGCIYWLPPAIF
PPAIFRSACSISVTYFPFDW----
*********

8. Low count correction using Blosum matrices
Every time for instance L/V is observed, I is also likely to occur
Estimate low (pseudo) count correction using this approach
As more data are included the pseudo count correction becomes less important
Blosum62 substitution frequencies
# I L V L V
Neff: Number of sequences
b: Weight on pseudo count or weight on prior

9. Information content Information and entropy Shannon information (D)
Conserved amino acid regions contain high degree of information (high order == low entropy)
Variable amino acid regions contain low degree of information (low order == high entropy)
Shannon information (D)
D = log2(N) + S pi log2 pi (for proteins N=20, DNA N=4)
Conserved residue pA=1, pi<>A=0,
D = log2(N) ( = 4.3 for proteins)
Variable region pA=0.05, pC=0.05, ..,
D = 0

10. Sequence logo Height of a column equal to D
MHC class I
High information
positions
Height of a column equal to D
Relative height of a letter is pA
Highly useful tool to visualize sequence motifs

11. More on logos Information content Shannon, qi = 1/N = 0.05
D = S pi log2 (pi/qi)
Shannon, qi = 1/N = 0.05
D = S pi log2 (pi) - S pi log2 (1/N)
= log2 N + S pi log2 (pi)
Kullback-Leibler, qi = background frequency
V/L/A more frequent than for instance C/H/W

12. Mutual information ALWGFFPVA ILKEPVHGV ILGFVFTLT LLFGYPVYV GLSPTVWLS
YMNGTMSQV
GILGFVFTL
WLSLLVPFV
FLPSDFFPS
P(G1) = 2/9 = 0.22, ..
P(V6) = 4/9 = 0.44,..
P(G1,V6) = 2/9 = 0.22,
P(G1)*P(V6) = 8/81 = 0.10
log(0.22/0.10) > 0

13. Mutual information
313 binding peptides
313 random peptides

14. Learning higher order correlation
Neural networks can learn higher order correlations!
What does this mean?
0 0 => 0
0 1 => 1
1 0 => 1
1 1 => 0
No linear function can learn this pattern

15. Learning higher order correlation
0 0 => 0; 1 0 => 1
1 1 => 0; 0 1 => 1 X1 X2 X1 X2
W11 W1 W2
W22
W21
W12 h1 hs V1 V2
Solution
Has no solution!

16. Take a deep breath Smile to you neighbor
End of first part
Take a deep breath
Smile to you neighbor

17. How to score a sequences to a probability matrix?
pij describes a motif
The probability that a peptide fits the motif is
The probability that the peptide fits a random model is
The ratio of the two gives the odds
The log gives the score

18. Weight matrices Wij = log(pij/qj)
Estimate amino acid frequencies from alignment including sequence weighting and pseudo counts
Construct a weight matrix as
Wij = log(pij/qj)
Here i is a position in the motif, and j an amino acid. qj is the prior (background) frequency for amino acid j.
W is a L x 20 matrix, L is motif length
Score sequences to weight matrix by looking up and adding L values from matrix

19. Weight matrix 2 What are log-odds scores (Wij = log(pij/qj))?
Does an monthly income of 2000 $ mean that you are rich?
Depends on where you live
In Denmark no
In Argentina yes
You must always compare your measured value (pij) to a background (qj)
In nature not all amino acids are found equally often
PA = 0.070, PW = 0.013
Finding 6% A is hence not significant, but 6% W highly significant

20. Scoring sequences to a weight matrix
A R N D C Q E G H I L K M F P S T W Y V
ILYQVPFSV
ALPYWNFAT
MTAQWWLDA
15.0
-3.4
0.8
Which peptide is most likely to bind?
Which peptide second?

21. Example from real life 10 peptides from MHCpep database
Bind to the MHC complex
Relevant for immune system recognition
Estimate sequence motif and weight matrix
Evaluate motif “correctness” on 528 peptides
ALAKAAAAM
ALAKAAAAN
ALAKAAAAR
ALAKAAAAT
ALAKAAAAV
GMNERPILT
GILGFVFTM
TLNAWVKVV
KLNEPVLLL
AVVPFIVSV

22. Example (cont.) Raw sequence counting No sequence weighting
ALAKAAAAM
ALAKAAAAN
ALAKAAAAR
ALAKAAAAT
ALAKAAAAV
GMNERPILT
GILGFVFTM
TLNAWVKVV
KLNEPVLLL
AVVPFIVSV
Raw sequence counting
No sequence weighting
No pseudo count
Prediction accuracy 0.45

23. Prediction accuracy
Pearson correlation 0.45

24. Example (cont.) Sequence weighting No pseudo count
ALAKAAAAM
ALAKAAAAN
ALAKAAAAR
ALAKAAAAT
ALAKAAAAV
GMNERPILT
GILGFVFTM
TLNAWVKVV
KLNEPVLLL
AVVPFIVSV
Example (cont.)
Sequence weighting
No pseudo count
Prediction accuracy 0.5

25. Example (cont.) Sequence weighting and pseudo count
ALAKAAAAM
ALAKAAAAN
ALAKAAAAR
ALAKAAAAT
ALAKAAAAV
GMNERPILT
GILGFVFTM
TLNAWVKVV
KLNEPVLLL
AVVPFIVSV
Example (cont.)
Sequence weighting and pseudo count
Prediction accuracy 0.60
Motif found on a large dataset
Prediction accuracy 0.79

26. Hidden Markov Models
Weight matrices do not deal with insertions and deletions
In alignments, this is done in an ad-hoc manner by optimization of the two gap penalties for first gap and gap extension
HMM is a natural frame work where insertions/deletions are dealt with explicitly

27. Why hidden?
The unfair casion: Loaded die p(6) = 0.5; switch fair to load:0.05; switch load to fair: 0.1
Model generates numbers
Do not tell which die was used
Alignment (decoding) can give the most probable solution/path (Viterby)
FFFFFFLLLLLL
Or most probable set of states
0.95
0.9
1:1/6
2:1/6
3:1/6
4:1/6
5:1/6
6:1/6
1:1/10
2:1/10
3:1/10
4:1/10
5:1/10
6:1/2
0.05
0.10
Fair
Loaded

28. HMM (a simple example) ACA---ATG TCAACTATC ACAC--AGC AGA---ATC
Example from A. Krogh
Core region defines the number of states in the HMM (red)
Insertion and deletion statistics are derived from the non-core part of the alignment (black)
ACA---ATG
TCAACTATC
ACAC--AGC
AGA---ATC
ACCG--ATC
Core of alignment

29. HMM construction ACA---ATG TCAACTATC ACAC--AGC AGA---ATC ACCG--ATC
5 matches. A, 2xC, T, G
5 transitions in gap region
C out, G out
A-C, C-T, T out
Out transition 3/5
Stay transition 2/5
ACA---ATG
TCAACTATC
ACAC--AGC
AGA---ATC
ACCG--ATC .4 A C G T .2 .4 .2 .2 .6 .6 A C G T .8 A C G T A C G T .8 A C G T 1 A C G T A C G T 1. 1. .4 1. 1. .8 .2 .8 .2 .2 .2 .2 .8
ACA---ATG 0.8x1x0.8x1x0.8x0.4x1x1x0.8x1x0.2 = 3.3x10-2

30. Align sequence to HMM
ACA---ATG 0.8x1x0.8x1x0.8x0.4x1x0.8x1x0.2 = 3.3x10-2
TCAACTATC 0.2x1x0.8x1x0.8x0.6x0.2x0.4x0.4x0.4x0.2x0.6x1x1x0.8x1x0.8 = x10-2
ACAC--AGC = 1.2x10-2
AGA---ATC = 3.3x10-2
ACCG--ATC = 0.59x10-2
Consensus:
ACAC--ATC = 4.7x10-2, ACA---ATC = 13.1x10-2
Exceptional:
TGCT--AGG = x10-2

31. Align sequence to HMM - Null model
Score depends strongly on length
Null model is a random model. For length L the score is 0.25L
Log-odds score for sequence S
Log( P(S)/0.25L)
Positive score means more likely than Null model
ACA---ATG = 4.9
TCAACTATC = 3.0
ACAC--AGC = 5.3
AGA---ATC = 4.9
ACCG--ATC = 4.6
Consensus:
ACAC--ATC = 6.7
ACA---ATC = 6.3
Exceptional:
TGCT--AGG = -0.97
Note!

32. Model decoding (Viterby) The unfair casino
Log model
-0.02
-0.05
1:-0.78
2:-0.78
3:-0.78
4:-0.78
5:-0.78
6:-0-78
1:-1
2:-1
3:-1
4:-1
5:-1
6:-0.3
-1.3
Example: -1
FFFFLLL
Fair
Loaded 1 2 4 5 6 F
-0.78
-1.58
-2.38
-3.18
-3.98
-4.78
-5.58 L
Null
-3.08
-3.88
-4.68
-5.13
-5.48

33. HMM’s and weight matrices
In the case of un-gapped alignments HMM’s become simple weight matrices
To achieve high performance, the emission frequencies are estimated using the techniques of
Sequence weighting
Pseudo counts

34. Profile HMM’s
Alignments based on conventional scoring matrices (BLOSUM62) scores all positions in a sequence in an equal manner
Some positions are highly conserved, some are highly variable (more than what is described in the BLOSUM matrix)
Profile HMM’s are ideal suited to describe such position specific variations

35. Profile HMM’s Core: Position with < 2 gaps Insertion Conserved
Deletion
ADDGSLAFVPSEF--SISPGEKIVFKNNAGFPHNIVFDEDSIPSGVDASKISMSEEDLLN
TVNGAI--PGPLIAERLKEGQNVRVTNTLDEDTSIHWHGLLVPFGMDGVPGVSFPG---I
-TSMAPAFGVQEFYRTVKQGDEVTVTIT-----NIDQIED-VSHGFVVVNHGVSME---I
IE--KMKYLTPEVFYTIKAGETVYWVNGEVMPHNVAFKKGIV--GEDAFRGEMMTKD---
-TSVAPSFSQPSF-LTVKEGDEVTVIVTNLDE------IDDLTHGFTMGNHGVAME---V
ASAETMVFEPDFLVLEIGPGDRVRFVPTHK-SHNAATIDGMVPEGVEGFKSRINDE----
TKAVVLTFNTSVEICLVMQGTSIV----AAESHPLHLHGFNFPSNFNLVDPMERNTAGVP
TVNGQ--FPGPRLAGVAREGDQVLVKVVNHVAENITIHWHGVQLGTGWADGPAYVTQCPI
Core: Position with < 2 gaps

36. HMM vs alignment Detailed description of core
ADDGSLAFVPSEF--SISPGEKIVFKNNAGFPHNIVFDEDSIPSGVDASKISMSEEDLLN
TVNGAI--PGPLIAERLKEGQNVRVTNTLDEDTSIHWHGLLVPFGMDGVPGVSFPG---I
-TSMAPAFGVQEFYRTVKQGDEVTVTIT-----NIDQIED-VSHGFVVVNHGVSME---I
IE--KMKYLTPEVFYTIKAGETVYWVNGEVMPHNVAFKKGIV--GEDAFRGEMMTKD---
-TSVAPSFSQPSF-LTVKEGDEVTVIVTNLDE------IDDLTHGFTMGNHGVAME---V
ASAETMVFEPDFLVLEIGPGDRVRFVPTHK-SHNAATIDGMVPEGVEGFKSRINDE----
TKAVVLTFNTSVEICLVMQGTSIV----AAESHPLHLHGFNFPSNFNLVDPMERNTAGVP
TVNGQ--FPGPRLAGVAREGDQVLVKVVNHVAENITIHWHGVQLGTGWADGPAYVTQCPI
Detailed description of core
Conserved/variable positions
Price for insertions/deletions varies at different locations in sequence
These features cannot be captured in conventional alignments

37. All M/D pairs must be visited once
Profile HMM’s
All M/D pairs must be visited once

38. Example Sequence profiles
Alignment of protein sequences 1PLC._ and 1GYC.A
E-value > 1000
Profile alignment
Align 1PLC._ against Swiss-prot
Make position specific weight matrix from alignment
Use this matrix to align 1PLC._ against 1GYC.A
E-value < Rmsd=3.3

39. Example continued Score = 97.1 bits (241), Expect = 9e-22 Rmsd=3.3 Å
Identities = 13/107 (12%), Positives = 27/107 (25%), Gaps = 17/107 (15%)
Query: 3 ADDGSLAFVPSEFSISPGEKI------VFKNNAGFPHNIVFDEDSIPSGVDASKIS 56
F G N G + +
Sbjct: VFPSPLITGKKGDRFQLNVVDTLTNHTMLKSTSIHWHGFFQAGTNWADGP 79
Query: 57 MSEEDLLNAKGETFEVAL---SNKGEYSFYCSP--HQGAGMVGKVTV 98
A G +F G G+ G V
Sbjct: 80 AFVNQCPIASGHSFLYDFHVPDQAGTFWYHSHLSTQYCDGLRGPFVV 126
Rmsd=3.3 Å
Model red
Template blue

40. TMHMM (trans-membrane HMM) (Sonnhammer, von Heijne, and Krogh)
Difference in amino acid composition. Easy in HMM.
Difficult in alignment.
Model TM length distribution.
Easy in HMM.
Difficult in alignment.

41. HMM packages NET-ID, HMMpro (http://www.netid.com/html/hmmpro.html)
HMMER (
S.R. Eddy, WashU St. Louis. Freely available.
SAM (
R. Hughey, K. Karplus, A. Krogh, D. Haussler and others, UC Santa Cruz. Freely available to academia, nominal license fee for commercial users.
META-MEME (
William Noble Grundy, UC San Diego. Freely available. Combines features of PSSM search and profile HMM search.
NET-ID, HMMpro (
Freely available to academia, nominal license fee for commercial users.
Allows HMM architecture construction.

42. trainanhmm 1.221 Copyright (C) 1998 by Anders Krogh
The unfair casion: Loaded die p(6) = 0.5; switch fair to load:0.05; switch load to fair: 0.1
Header {alphabet ;}
begin {
trans Fair:0.5 Loaded:0.5; }
Fair {
trans Fair:0.95 Loaded:0.05;
Loaded {
trans Fair:0.1 Loaded:0.9;
letter 6:0.5;
0.95
0.9
1:1/6
2:1/6
3:1/6
4:1/6
5:1/6
6:1/6
1:1/10
2:1/10
3:1/10
4:1/10
5:1/10
6:1/2
0.05
0.10
Fair
Loaded