Hidden Markov Models 1 2 K … 1 2 K … 1 2 K … … … … 1 2 K … x1x1 x2x2 x3x3 xKxK 2 1 K 2

CS262 Lecture 7, Win07, Batzoglou Variants of HMMs

CS262 Lecture 7, Win07, Batzoglou Higher-order HMMs How do we model “memory” larger than one time point? P(  i+1 = l |  i = k)a kl P(  i+1 = l |  i = k,  i -1 = j)a jkl … A second order HMM with K states is equivalent to a first order HMM with K 2 states state Hstate T a HT (prev = H) a HT (prev = T) a TH (prev = H) a TH (prev = T) state HHstate HT state THstate TT a HHT a TTH a HTT a THH a THT a HTH

CS262 Lecture 7, Win07, Batzoglou Similar Algorithms to 1 st Order P(  i+1 = l |  i = k,  i -1 = j)  V lk (i) = max j { V kj (i – 1) + … }  Time? Space?

CS262 Lecture 7, Win07, Batzoglou Modeling the Duration of States Length distribution of region X: E[l X ] = 1/(1-p) Geometric distribution, with mean 1/(1-p) This is a significant disadvantage of HMMs Several solutions exist for modeling different length distributions XY 1-p 1-q pq

CS262 Lecture 7, Win07, Batzoglou Example: exon lengths in genes

CS262 Lecture 7, Win07, Batzoglou Solution 1: Chain several states XY 1-p 1-q p q X X Disadvantage: Still very inflexible l X = C + geometric with mean 1/(1-p)

CS262 Lecture 7, Win07, Batzoglou Solution 2: Negative binomial distribution Duration in X: m turns, where  During first m – 1 turns, exactly n – 1 arrows to next state are followed  During m th turn, an arrow to next state is followed m – 1 P(l X = m) = n – 1 (1 – p) n-1+1 p (m-1)-(n-1) = n – 1 (1 – p) n p m-n X (n) p X (2) X (1) p 1 – p p …… Y 1 – p

CS262 Lecture 7, Win07, Batzoglou Example: genes in prokaryotes EasyGene: Prokaryotic gene-finder Larsen TS, Krogh A Negative binomial with n = 3

CS262 Lecture 7, Win07, Batzoglou Solution 3:Duration modeling Upon entering a state: 1.Choose duration d, according to probability distribution 2.Generate d letters according to emission probs 3.Take a transition to next state according to transition probs Disadvantage: Increase in complexity of Viterbi: Time: O(D) Space: O(1) where D = maximum duration of state F d<D f x i …x i+d-1 PfPf Warning, Rabiner’s tutorial claims O(D 2 ) & O(D) increases

CS262 Lecture 7, Win07, Batzoglou Viterbi with duration modeling Recall original iteration: Vl(i) = max k V k (i – 1) a kl  e l (x i ) New iteration: V l (i) = max k max d=1…Dl V k (i – d)  P l (d)  a kl   j=i-d+1…i e l (x j ) FL transitions emissions d<D f x i …x i + d – 1 emissions d<D l x j …x j + d – 1 PfPf PlPl Precompute cumulative values

CS262 Lecture 7, Win07, Batzoglou Proteins, Pair HMMs, and Alignment

CS262 Lecture 7, Win07, Batzoglou A state model for alignment -AGGCTATCACCTGACCTCCAGGCCGA--TGCCC--- TAG-CTATCAC--GACCGC-GGTCGATTTGCCCGACC IMMJMMMMMMMJJMMMMMMJMMMMMMMIIMMMMMIII M (+1,+1) I (+1, 0) J (0, +1) Alignments correspond 1-to-1 with sequences of states M, I, J

CS262 Lecture 7, Win07, Batzoglou Let’s score the transitions -AGGCTATCACCTGACCTCCAGGCCGA--TGCCC--- TAG-CTATCAC--GACCGC-GGTCGATTTGCCCGACC IMMJMMMMMMMJJMMMMMMJMMMMMMMIIMMMMMIII M (+1,+1) I (+1, 0) J (0, +1) Alignments correspond 1-to-1 with sequences of states M, I, J s(x i, y j ) -d -e

CS262 Lecture 7, Win07, Batzoglou Alignment with affine gaps – state version Dynamic Programming: M(i, j):Optimal alignment of x 1 …x i to y 1 …y j ending in M I(i, j): Optimal alignment of x 1 …x i to y 1 …y j ending in I J(i, j): Optimal alignment of x 1 …x i to y 1 …y j ending in J The score is additive, therefore we can apply DP recurrence formulas

CS262 Lecture 7, Win07, Batzoglou Alignment with affine gaps – state version Initialization: M(0,0) = 0; M(i, 0) = M(0, j) = - , for i, j > 0 I(i,0) = d + i  e;J(0, j) = d + j  e Iteration: M(i – 1, j – 1) M(i, j) = s(x i, y j ) + max I(i – 1, j – 1) J(i – 1, j – 1) e + I(i – 1, j) I(i, j) = max d + M(i – 1, j) e + J(i, j – 1) J(i, j) = max d + M(i, j – 1) Termination: Optimal alignment given by max { M(m, n), I(m, n), J(m, n) }

CS262 Lecture 7, Win07, Batzoglou Brief introduction to the evolution of proteins Protein sequence and structure Protein classification Phylogeny trees Substitution matrices

CS262 Lecture 7, Win07, Batzoglou Muscle cells and contraction

CS262 Lecture 7, Win07, Batzoglou Actin and myosin during muscle movement

CS262 Lecture 7, Win07, Batzoglou Actin structure

CS262 Lecture 7, Win07, Batzoglou Actin sequence Actin is ancient and abundant  Most abundant protein in cells  1-2 actin genes in bacteria, yeasts, amoebas  Humans: 6 actin genes  -actin in muscles;  -actin,  -actin in non-muscle cells ~4 amino acids different between each version MUSCLE ACTIN Amino Acid Sequence 1 EEEQTALVCD NGSGLVKAGF AGDDAPRAVF PSIVRPRHQG VMVGMGQKDS YVGDEAQSKR 61 GILTLKYPIE HGIITNWDDM EKIWHHTFYN ELRVAPEEHP VLLTEAPLNP KANREKMTQI 121 MFETFNVPAM YVAIQAVLSL YASGRTTGIV LDSGDGVSHN VPIYEGYALP HAIMRLDLAG 181 RDLTDYLMKI LTERGYSFVT TAEREIVRDI KEKLCYVALD FEQEMATAAS SSSLEKSYEL 241 PDGQVITIGN ERFRGPETMF QPSFIGMESS GVHETTYNSI MKCDIDIRKD LYANNVLSGG 301 TTMYPGIADR MQKEITALAP STMKIKIIAP PERKYSVWIG GSILASLSTF QQMWITKQEY 361 DESGPSIVHR KCF

CS262 Lecture 7, Win07, Batzoglou A related protein in bacteria

CS262 Lecture 7, Win07, Batzoglou Relation between sequence and structure

CS262 Lecture 7, Win07, Batzoglou Protein Phylogenies Proteins evolve by both duplication and species divergence

CS262 Lecture 7, Win07, Batzoglou Protein Phylogenies – Example

CS262 Lecture 7, Win07, Batzoglou Structure Determines Function What determines structure? Energy Kinematics How can we determine structure? Experimental methods Computational predictions The Protein Folding Problem

CS262 Lecture 7, Win07, Batzoglou Primary Structure: Sequence The primary structure of a protein is the amino acid sequence

CS262 Lecture 7, Win07, Batzoglou Primary Structure: Sequence Twenty different amino acids have distinct shapes and properties

CS262 Lecture 7, Win07, Batzoglou Primary Structure: Sequence A useful mnemonic for the hydrophobic amino acids is "FAMILY VW"

CS262 Lecture 7, Win07, Batzoglou Secondary Structure: , , & loops  helices and  sheets are stabilized by hydrogen bonds between backbone oxygen and hydrogen atoms

CS262 Lecture 7, Win07, Batzoglou Tertiary Structure: A Protein Fold

CS262 Lecture 7, Win07, Batzoglou PDB Growth New PDB structures

CS262 Lecture 7, Win07, Batzoglou Only a few folds are found in nature

CS262 Lecture 7, Win07, Batzoglou Protein classification Number of protein sequences grows exponentially Number of solved structures grows exponentially Number of new folds identified very small (and close to constant) Protein classification can  Generate overview of structure types  Detect similarities (evolutionary relationships) between protein sequences  Help predict 3D structure of new protein sequences SCOP release 1.71, Class# folds# superfamilies# families All alpha proteins All beta proteins Alpha and beta proteins (a/b) Alpha and beta proteins (a+b) Multi-domain proteins48 64 Membrane & cell surface Small proteins Total Classification of 27,599 protein structures in PDB

CS262 Lecture 7, Win07, Batzoglou Protein world Protein fold Protein structure classification Protein superfamily Protein family Morten Nielsen,CBS, BioCentrum, DTU

CS262 Lecture 7, Win07, Batzoglou Structure Classification Databases SCOP  Manual classification (A. Murzin)  scop.berkeley.edu scop.berkeley.edu CATH  Semi manual classification (C. Orengo)  FSSP  Automatic classification (L. Holm)  Morten Nielsen,CBS, BioCentrum, DTU

CS262 Lecture 7, Win07, Batzoglou Major classes in SCOP Classes  All  proteins  All  proteins   and  proteins (  /  )   and  proteins (  +  )  Multi-domain proteins  Membrane and cell surface proteins  Small proteins  Coiled coil proteins Morten Nielsen,CBS, BioCentrum, DTU

CS262 Lecture 7, Win07, Batzoglou All  : Hemoglobin (1bab) Morten Nielsen,CBS, BioCentrum, DTU

CS262 Lecture 7, Win07, Batzoglou All  : Immunoglobulin (8fab) Morten Nielsen,CBS, BioCentrum, DTU

CS262 Lecture 7, Win07, Batzoglou  Triosephosphate isomerase (1hti) Morten Nielsen,CBS, BioCentrum, DTU

CS262 Lecture 7, Win07, Batzoglou  : Lysozyme (1jsf) Morten Nielsen,CBS, BioCentrum, DTU

CS262 Lecture 7, Win07, Batzoglou Families Proteins whose evolutionarily relationship is readily recognizable from the sequence (>~25% sequence identity) Families are further subdivided into Proteins Proteins are divided into Species  The same protein may be found in several species Fold Family Superfamily Proteins Morten Nielsen,CBS, BioCentrum, DTU

CS262 Lecture 7, Win07, Batzoglou Superfamilies Proteins which are (remotely) evolutionarily related  Sequence similarity low  Share function  Share special structural features Relationships between members of a superfamily may not be readily recognizable from the sequence alone Fold Family Superfamily Proteins Morten Nielsen,CBS, BioCentrum, DTU

CS262 Lecture 7, Win07, Batzoglou Folds >~50% secondary structure elements arranged in the same order in sequence and in 3D No evolutionary relation Fold Family Superfamily Proteins Morten Nielsen,CBS, BioCentrum, DTU

CS262 Lecture 7, Win07, Batzoglou Substitutions of Amino Acids Mutation rates between amino acids have dramatic differences!

CS262 Lecture 7, Win07, Batzoglou Substitution Matrices BLOSUM matrices: 1.Start from BLOCKS database (curated, gap-free alignments) 2.Cluster sequences according to > X% identity 3.Calculate A ab : # of aligned a-b in distinct clusters, correcting by 1/mn, where m, n are the two cluster sizes 4.Estimate P(a) = (  b A ab )/(  c≤d A cd ); P(a, b) = A ab /(  c≤d A cd )

CS262 Lecture 7, Win07, Batzoglou Probabilistic interpretation of an alignment An alignment is a hypothesis that the two sequences are related by evolution Goal: Produce the most likely alignment Assert the likelihood that the sequences are indeed related

CS262 Lecture 7, Win07, Batzoglou A Pair HMM for alignments M P(x i, y j ) I P(x i ) J P(y j ) 1 – 2  1 –      This model generates two sequences simultaneously Match/Mismatch state M: P(x, y) reflects substitution frequencies between pairs of amino acids Insertion states I, J: P(x), P(y) reflect frequencies of each amino acid  : set so that 1/2  is avg. length before next gap  : set so that 1/(1 –  ) is avg. length of a gap M Model M optional

CS262 Lecture 7, Win07, Batzoglou A Pair HMM for unaligned sequences I P(x i ) J P(y j ) 11 Two sequences are independently generated from one another P(x, y | R) = P(x 1 )…P(x m ) P(y 1 )…P(y n ) =  i P(x i )  j P(y j ) R Model R

CS262 Lecture 7, Win07, Batzoglou To compare ALIGNMENT vs. RANDOM hypothesis Every pair of letters contributes:M (1 – 2  ) P(x i, y j ) when matched  P(x i ) P(y j ) when gappedR P(x i ) P(y j ) in random model Focus on comparison of P(x i, y j ) vs. P(x i ) P(y j ) M P(x i, y j ) I P(x i ) J P(y j ) 1 – 2  1 –     I P(x i ) J P(y j ) 1 1

CS262 Lecture 7, Win07, Batzoglou To compare ALIGNMENT vs. RANDOM hypothesis Every pair of letters contributes:M (1 – 2  ) P(x i, y j ) when matched  P(x i ) P(y j ) when gappedR P(x i ) P(y j ) in random model Focus on comparison of P(x i, y j ) vs. P(x i ) P(y j ) M P(x i, y j ) I P(x i ) J P(y j ) 1 – 2   (1 –  ) (1 – 2  )  I P(x i ) J P(y j ) – 2  Equivalent!

CS262 Lecture 7, Win07, Batzoglou To compare ALIGNMENT vs. RANDOM hypothesis Idea: We will divide alignment score by the random score, and take logarithms Let P(x i, y j ) s(x i, y j ) = log ––––––––– + log (1 – 2  ) P(x i ) P(y j )  (1 –  ) P(x i ) d = – log ––––––––––––– (1 – 2  ) P(x i )  P(x i ) e = – log –––––– P(x i ) = Defn substitution score = Defn gap initiation penalty = Defn gap extension penalty

CS262 Lecture 7, Win07, Batzoglou The meaning of alignment scores The Viterbi algorithm for Pair HMMs corresponds exactly to global alignment DP with affine gaps V M (i, j) = max { V M (i – 1, j – 1), V I ( i – 1, j – 1) – d, V j ( i – 1, j – 1) } + s(x i, y j ) V I (i, j) = max { V M (i – 1, j) – d, V I ( i – 1, j) – e } V J (i, j) = max { V M (i – 1, j) – d, V I ( i – 1, j) – e }  s(.,.) (1 – 2  ) ~how often a pair of letters substitute one another   1/mean length of next gap   (1 –  ) / (1 – 2  ) 1/mean arrival time of next gap

CS262 Lecture 7, Win07, Batzoglou The meaning of alignment scores Match/mismatch scores: P(x i, y j ) s(a, b)  log –––––––––– (ignore log(1 – 2  ) for the moment) P(x i ) P(y j ) Example: DNA regions between human and mouse genes have average conservation of 80% 1.What is the substitution score for a match? P(a, a) + P(c, c) + P(g, g) + P(t, t) = 0.8  P(x, x) = 0.2 P(a) = P(c) = P(g) = P(t) = 0.25 s(x, x) = log [ 0.2 / ] = What is the substitution score for a mismatch? P(a, c) +…+P(t, g) = 0.2  P(x, y  x) = 0.2/12 = s(x, y  x) = log[ / ] = What ratio matches/(matches + mism.) gives score 0? x(#match) – y(#mism) = (#match) – (#mism) = 0 #match = 1.137(#mism) matches = 53.2%

CS262 Lecture 7, Win07, Batzoglou Substitution Matrices BLOSUM matrices: 1.Start from BLOCKS database (curated, gap-free alignments) 2.Cluster sequences according to > X% identity 3.Calculate A ab : # of aligned a-b in distinct clusters, correcting by 1/mn, where m, n are the two cluster sizes 4.Estimate P(a) = (  b A ab )/(  c≤d A cd ); P(a, b) = A ab /(  c≤d A cd )

CS262 Lecture 7, Win07, Batzoglou BLOSUM matrices BLOSUM 50 BLOSUM 62 (The two are scaled differently)