1. Class review Sushmita Roy BMI/CS 576 www.biostat.wisc.edu/bmi576/ sroy@biostat.wisc.edu Dec 11 th, 2014

2. What you should know Markov models and Hidden Markov models – Forward, Viterbi and backward algorithms – Parameter estimation Clustering – Hierarchical, flat, model-based (Gaussian mixture models) Network modeling and analysis – Bayesian networks vs dependency networks – Network reconstruction algorithms – Properties of networks Phylogenetic trees – Distance-based methods: Neighbor Joining, UPGMA – Parsimony methods: Weighted Parsimony algorithm – Probabilistic methods: Felsenstein algorithm Sequence alignment – Global and local alignment – Pairwise and multiple sequence alignment

3. A Markov chain model for DNA sequence A TC begin state transition.34.16.38.12 transition probabilities G

4. Estimating parameters of a Markov chain Parameters of a Markov chain are the transition probabilities The parameters that need to be estimated are determined by the structure of the model Laplace correction needs to be applied (if asked) only to the parameters that need to be estimated

5. Homework 4: 1a ACGEnd Begin4/62/6 A3/142/149/14 C4/113/111/113/11 G4/16 5/163/16 Don’t need to estimate these parameters 1.Probabilities associated with outgoing transitions must sum to 1 2.Laplace correction for “Begin” adds a count of 2 to the denominator 3.Laplace correction for “A” adds 3 to the denominator 4.Laplace correction for “G” and “C” adds 4 to the denominator

6. Hidden Markov models The learning problems Forward algorithm: – Compute the likelihood of an observed sequence Viterbi algorithm – Find the most likely state assignment Baum-Welch algorithm – E-step Forward algorithm Backward algorithm – M-step Parameter estimation based on expected counts of transitions and emissions

7. Formally defining a HMM States Emission alphabet Parameters – State transition probabilities for probabilistic transitions from state at time t to state at time t+1 – Emission probabilities for probabilistically emitting symbols from a state

8. An example HMM 0.8 probability of emitting character A in state 2 probability of a transition from state 1 to state 3 0.4 A 0.4 C 0.1 G 0.2 T 0.3 A 0.1 C 0.4 G 0.4 T 0.1 A 0.2 C 0.3 G 0.3 T 0.2 beginend 0.5 0.2 0.8 0.6 0.1 0.9 0.2 05 4 3 2 1 A 0.4 C 0.1 G 0.1 T 0.4

9. Three important questions in HMMs How likely is an HMM to have generated a given sequence? – Forward algorithm What is the most likely “path” for generating a sequence of observations – Viterbi algorithm How can we learn an HMM from a set of sequences? – Forward-backward or Baum-Welch (an EM algorithm)

10. Reviewing the notation States with emissions will be numbered from 1 to K – 0 begin state, N end state observed character at position t Observed sequence Hidden state sequence or path Transition probabilities Emission probabilities: Probability of emitting symbol b from state k

11. How likely is a given sequence: Forward algorithm Define as the probability of observing and ending in state k at time t This can be written recursively as follows

12. Steps of the Forward algorithm Initialization: denote 0 for the “begin” state Recursion: for t=1 to T Termination

13. Learning without hidden information Learning is simple if we know the correct path for each sequence in our training set Estimate parameters by counting the number of times each parameter is used across the training set 5 C A G T 02244 begin end 0 4 3 2 1 5

14. Learning without hidden information Transition probabilities Emission probabilities Number of transitions from state k to state l Number of times c is emitted from k

15. Learning with hidden information 5 C A G T 0 begin end 0 4 3 2 1 5 ???? if we don’t know the correct path for each sequence in our training set, consider all possible paths for the sequence estimate parameters through a procedure that counts the expected number of times each parameter is used across the training set

16. Learning HMM parameters with the Baum-Welch algorithm algorithm sketch: – initialize parameters of model – iterate until convergence calculate the expected number of times each transition or emission is used adjust the parameters to maximize the likelihood of these expected values

17. The expectation step We need to know the probability of the symbol at t being produced by state k, given the entire sequence x Given these we can compute our expected counts for state transitions, character emissions We also need to know the probability of symbol at t and (t+1) being produced by state k, and l respectively given sequence x

18. Computing First we compute the probability of the entire observed sequence with the t th symbol being generated by state k Then our quantity of interest is computed as Obtained from the forward algorithm

19. To compute We need the forward and backward algorithm Forward algorithm f k (t) Backward algorithm b k (t) Computing

20. Steps of the backward algorithm Initialization ( t=T ) Recursion ( t=T-1 to 1 ) Termination Note, the same quantity can be obtained from the forward algorithm as well

21. Computing Using the forward and backward variables, this is computed as

22. Computing This is the probability of symbols at t and t+1 emitted from states k and l given the entire sequence x

23. Putting it all together Assume we are given J training instances x 1,..,x j,.. x J Expectation step – Using current parameter values compute for each x j Apply the forward and backward algorithms Compute – expected number of transitions between all pairs of states – expected number of emissions for all states Maximization step – Using current expected counts Compute the transition and emission probabilities

24. The expectation step: emission count We need the expected number of times c is emitted by state k x j : j th training sequences sum over positions where c occurs in x

25. The expectation step: transition count Expected number of times of transitions from k to l

26. The maximization step Estimate new emission parameters by: Just like in the simple case but typically we’ll do some “smoothing” (e.g. add pseudocounts) Estimate new transition parameters by

27. The Baum-Welch algorithm initialize the parameters of the HMM iterate until convergence – initialize, with pseudocounts – E-step: for each training set sequence j = 1…n calculate values for sequence j add the contribution of sequence j to, – M-step: update the HMM parameters using,

28. Baum-Welch algorithm example Given – The HMM with the parameters initialized as shown – Two training sequences TAG, ACG A 0.1 C 0.4 G 0.4 T 0.1 A 0.4 C 0.1 G 0.1 T 0.4 beginend 1.0 0.1 0.90.2 0.8 0 3 21 we’ll work through one iteration of Baum-Welch

29. Baum-Welch example (cont) Determining the forward values for TAG Here we compute just the values that are needed for computing successive values. For example, no point in calculating f 2 (1) In a similar way, we also compute forward values for ACG

30. Baum-Welch example (cont) Determining the backward values for TAG Again, here we compute just the values that are needed In a similar way, we also compute backward values for ACG

31. Baum-Welch example (cont) determining the expected emission counts for state 1 contribution of TAG contribution of ACG *note that the forward/backward values in these two columns differ; in each column they are computed for the sequence associated with the column

32. Baum-Welch example (cont) Determining the expected transition counts (not using pseudocounts) Contribution of TAG + Contribution of ACG

33. Baum-Welch example (cont) Maximization step: determining probabilities for state 1

34. Clustering Find groups of entities that exhibit similar attributes Hierarchical clustering – Different types of linkage – Obtaining a flat clustering from a dendrogram Flat clustering – K-means – Gaussian mixture Key issues to think about – How to pick k, or how to decide where to cut in hierarchical clustering – Validation and interpretation of clusters – Distance metric to define dissimilarity between entities

35. K -means algorithm Input: K, number of clusters, a set X={x 1,.. x n } of data points, where x i are p -dimensional vectors Initialize – Select initial cluster means Repeat until convergence – Assign each x i to cluster C(i) such that – Re-estimate the mean of each cluster based on new members

36. K -means clustering consider an example in which our vectors have 2 dimensions ++ + + profile cluster center

37. K -means clustering each iteration involves two steps – assignment of profiles to clusters – re-computation of the means ++ + + ++ + + assignment re-computation of means

38. K -means: updating the mean To compute the mean of the c th cluster Number of genes in cluster c All genes in cluster c

39. K -means stopping criteria Assignment of objects to clusters don’t change Fix the max number of iterations Optimization criterion changes by a small value

40. Gaussian mixture model based clustering K -means is hard clustering – At each iteration, a datapoint is assigned to one and only one cluster We can do soft clustering based on Gaussian mixture models – Each cluster is represented by a distribution (in our case a Gaussian) – We assume the data is generated by a mixture of the Gaussians

41. Gaussian mixture model clustering A model-based clustering approach For K clusters, we will have K Gaussians The Gaussian mixture model describes the probability density of a data point x as Clustering entails learning a Gaussian mixture model for the data to be clustered Prior probability of the c th Gaussian c th Gaussian

42. Learning a Gaussian mixture model (GMM) Assume we have N training data points (e.g. genes) And we know what K is Parameters of the GMM are It is common to ignore the off-diagonal elements. For example for a 2-dimensional Gaussian we have: Mixture probabilitiesMeans Co-variances

43. Learning a Gaussian mixture model (GMM) If we knew the cluster assignments estimating means and variances is easy – Take the data points in cluster c and estimate parameters for the Gaussian from cluster c But we don’t. We will use the expectation-maximization (EM) algorithm to learn GMM parameters Recall the EM algorithm is useful when we have hidden variables What are the hidden variables here? – Cluster assignments

44. Expectation step We would like to estimate the probability of Z ic =1 – c th Gaussian generating data point x i That is We will use to denote We can think of as the contribution of each data point x i to cluster c

45. Maximization step Here we need to estimate the parameters for each Gaussian And the mixing weights Variance for the r th dimension

46. Putting the E and M steps all together Initialize using initial partitions of the data Repeat until convergence – Expectation step Compute the for each data point and Gaussian – Maximization step – Use to update for all c=1 to K – Compute likelihood to check for convergence

47. GMM clustering example Consider a one-dimensional clustering problem in which the data given are: x 1 = -4 x 2 = -3 x 3 = -1 x 4 = 3 x 5 = 5 Assume number of Gaussians, K=2. So we have parameters: Assume initials parameters as: Their density functions are is: Gaussian 1 Gaussian 2

48. GMM clustering example

49. GMM clustering example: E-step

50. GMM clustering example: M-step

51. GMM clustering example Here we have shown just one step of the EM procedure We would continue the E- and M-steps until convergence

52. Comparing K-means and GMMs K-means – Hard clustering – Optimizes within cluster scatter – Requires estimation of means GMMs – Soft clustering – Optimizes likelihood of data – Requires estimation of mean and covariance and mixture probabilities

53. Networks Different types of molecular networks Network reconstruction – Algorithms vary depending upon how they represent the relationship between nodes Bayesian networks – Representing biological networks as Bayesian network – Types of conditional distributions – Parameter and structure learning from data Network analysis – What properties characterize complex networks Degree distributions Centrality Network motifs Modularity Network applications – Interpreting gene sets – Connecting two gene sets

54. Bayesian networks (BN) A special type of probabilistic graphical model Has two parts: – A graph which is directed and acyclic – A set of conditional distributions Directed Acyclic Graph (DAG) – The nodes denote random variables X 1 … X N – The edges encode statistical dependencies between the random variables Establish parent child relationships Each node X i has a conditional probability distribution (CPD) representing P(X i | Parents(X i ) ) Provides a tractable way to represent large joint distributions

55. An example Bayesian network Cloudy (C) Rain (R) Sprinkler (S) Adapted from Kevin Murphy: Intro to Graphical models and Bayes networks: http://www.cs.ubc.ca/~murphyk/Bayes/bnintro.html WetGrass (W) P(C=F) P(C=T) 0.5 P(R=F) P(R=T) 0.8 0.2 T 0.2 0.8 P(S=F) P(S=T) 0.5 F T 0.9 0.1 P(W=F) P(W=T) 1 0 F T F 0.1 0.9 C F T T 0.1 0.9 0.01 0.99 C F S R

56. Bayesian network representation of a transcriptional network HSP12 Sko1 Hot1 Sko1 HSP12 Random variables encode expression levels T ARGET ( CHILD ) R EGULATORS (P ARENTS ) X1X1 X2X2 X3X3 X1X1 X2X2 X3X3 P(X 3 |X 1,X 2 ) GenesRandom variables P(X 2 ) P(X 1 ) An example Bayesian network Assume HSP12’s expression is dependent upon Hot1 and Sko1 binding to HSP12’s promoter HSP12 ON HSP12 Sko1 HSP12 OFF

57. Bayesian networks compactly represent joint distributions CPD

58. Example Bayesian network of 5 variables X1X1 X2X2 X3X3 X5X5 X4X4 P(X 3 |X 1,X 2 ) P(X 2 ) P(X 1 ) P(X 4 ) P(X 5 |X 3, X 4 )

59. CPD in Bayesian networks CPD: Conditional probability distributions are central to Bayesian networks We have a CPD for each random variable in our graph CPDs describe the distribution of a Child variable given the state of its parents. The same structure can be parameterized in different ways For example for discrete variables we can have table or tree representations

60. Consider the following case with Boolean variables X 1, X 2, X 3, X 4 where X 1, X 2 and X 3 are the parents of X 4 Representing CPDs as tables X1X1 X2X2 X3X3 tf ttt0.90.1 ttf0.90.1 tft0.90.1 tff0.90.1 ftt0.80.2 ftf0.5 fft fff P( X 4 | X 1, X 2, X 3 ) as a table X4X4 X1X1 X2X2 X4X4 X3X3 P( X 4 | X 1, X 2, X 3 )

61. A tree representation of a CPD P( X 4 | X 1, X 2, X 3 ) as a tree P(X 4 = t ) = 0.9 X1X1 f t X2X2 P(X 4 = t ) = 0.5 ft X3X3 P(X 4 = t ) = 0.8 ft X1X1 X2X2 X4X4 X3X3 Allows more compact representation of CPDs. For example, we can ignore some quantities.

62. The learning problems Parameter learning on known structure – Given training data estimate parameters of the CPDs Structure learning – Given training data, find the statistical dependency structure, and that best describe – Subsumes parameter learning For every candidate graph, we need to estimate the parameters

63. Example of estimating CPD table from data Consider the four random variables X 1, X 2, X 3, X 4 Assume we observe the following samples of assignments to these variables To estimate P(X 4 |X 1,X 2,X 3 ), we need consider all configurations of X 1,X 2, X 3 and estimate the probability of X 4 being T or F TFTT TTFT TTFT TFTT TFTF TFTF FFTF X1X1 X2X2 X3X3 X4X4 For example, consider X 1 =T, X 2 =F, X 3 =T P(X 4 =T|X 1 =T, X 2 =F, X 3 =T)=2/4 P(X 4 =F|X 1 =T, X 2 =F, X 3 =T)=2/4

64. Structure learning using score-based search... Bayesian network Maximum likelihood parameters Data

65. Scoring a Bayesian network The score of a Bayesian network (BN) is determined by how well the BN describes the data This in turn is a function of the data likelihood Given data The score of a BN is therefore Parents of X i Assignment to parents of X i in the d th sample

66. Scoring a Bayesian network Score of a graph G decomposes over individual variables Which can be re-arranged to be written as the outer sum over variables This enables us to efficiently compute the score effect of local changes – That is changes to the parent set of individual random variables

67. Heuristic search of Bayesian network structures Make local operations to the graph structure – Add an edge – Delete an edge – Reverse an edge Evaluate score and select the network configuration with best score We just need to check for cycles Working with gene expression data requires additional considerations – Reduce potential parents: statistically or using biological knowledge – Bootstrap based confidence estimation – Permutation based assessment of confidence

68. Bayesian network vs Module network Each variable takes three values: UP, DOWN, SAME

69. Bayesian network vs Module network Bayesian network – Different CPD per random variable – Learning only requires to search for parents Module network – CPD per module Same CPD for all random variables in the same module – Learning requires parent search and module membership assignment

70. Some comments about expression-based network inference methods We have seen two types of algorithms to learn these networks – Per-gene methods Sparse candidate: learn regulators for individual genes GENIE3 – Per-module methods Module networks: learn regulators for sets of genes/modules – Other implementations of module networks exist LIRNET: Learning a Prior on Regulatory Potential from eQTL Data – Su In Lee et al, Plos genetics 2009 (http://www.plosgenetics.org/article/info%3Adoi%2F10.1371%2F journal.pgen.1000358) LeMoNe: Learning Module Networks – Michoel et al 2007 (http://www.biomedcentral.com/1471- 2105/8/S2/S5)

71. Phylogenetic trees Phylogenetic tree construction – Distance-based methods UPGMA Neighbor joining – Parsimony methods Weighted parsimony to find the minimal cost tree – Probabilistic methods Felsenstein’s algorithm to compute the probability of observations at leaf nodes Scoring a given tree versus searching the space of trees