. Computational Genomics Lecture 7c Hidden Markov Models (HMMs) © Ydo Wexler & Dan Geiger (Technion) and by Nir Friedman (HU) Modified by Benny Chor (TAU)

Slides:



Advertisements
Similar presentations
1 Introduction to Discrete-Time Markov Chain. 2 Motivation  many dependent systems, e.g.,  inventory across periods  state of a machine  customers.
Advertisements

. Markov Chains. 2 Dependencies along the genome In previous classes we assumed every letter in a sequence is sampled randomly from some distribution.
Hidden Markov Models (1)  Brief review of discrete time finite Markov Chain  Hidden Markov Model  Examples of HMM in Bioinformatics  Estimations Basic.
CS433 Modeling and Simulation Lecture 06 – Part 03 Discrete Markov Chains Dr. Anis Koubâa 12 Apr 2009 Al-Imam Mohammad Ibn Saud University.
Flows and Networks Plan for today (lecture 2): Questions? Continuous time Markov chain Birth-death process Example: pure birth process Example: pure death.
Markov Chains Modified by Longin Jan Latecki
IERG5300 Tutorial 1 Discrete-time Markov Chain
Markov Chains 1.
. Markov Chains as a Learning Tool. 2 Weather: raining today40% rain tomorrow 60% no rain tomorrow not raining today20% rain tomorrow 80% no rain tomorrow.
1 Markov Chains (covered in Sections 1.1, 1.6, 6.3, and 9.4)
Hidden Markov Models Tunghai University Fall 2005.
. Computational Genomics Lecture 10 Hidden Markov Models (HMMs) © Ydo Wexler & Dan Geiger (Technion) and by Nir Friedman (HU) Modified by Benny Chor (TAU)
. Hidden Markov Models - HMM Tutorial #5 © Ydo Wexler & Dan Geiger.
Lecture 8: Hidden Markov Models (HMMs) Michael Gutkin Shlomi Haba Prepared by Originally presented at Yaakov Stein’s DSPCSP Seminar, spring 2002 Modified.
Topics Review of DTMC Classification of states Economic analysis
11 - Markov Chains Jim Vallandingham.
Lecture 12 – Discrete-Time Markov Chains
. Computational Genomics Lecture 10 Hidden Markov Models (HMMs) © Ydo Wexler & Dan Geiger (Technion) and by Nir Friedman (HU) Modified by Benny Chor (TAU)
G12: Management Science Markov Chains.
Chapter 17 Markov Chains.
Flows and Networks (158052) Richard Boucherie Stochastische Operations Research -- TW wwwhome.math.utwente.nl/~boucherierj/onderwijs/158052/ html.
Андрей Андреевич Марков. Markov Chains Graduate Seminar in Applied Statistics Presented by Matthias Theubert Never look behind you…
Ch-9: Markov Models Prepared by Qaiser Abbas ( )
Entropy Rates of a Stochastic Process
Hidden Markov Models Fundamentals and applications to bioinformatics.
Tutorial 8 Markov Chains. 2  Consider a sequence of random variables X 0, X 1, …, and the set of possible values of these random variables is {0, 1,
6.896: Probability and Computation Spring 2011 Constantinos (Costis) Daskalakis lecture 2.
Overview of Markov chains David Gleich Purdue University Network & Matrix Computations Computer Science 15 Sept 2011.
. Hidden Markov Model Lecture #6. 2 Reminder: Finite State Markov Chain An integer time stochastic process, consisting of a domain D of m states {1,…,m}
Markov Chains Lecture #5
048866: Packet Switch Architectures Dr. Isaac Keslassy Electrical Engineering, Technion Review.
Hidden Markov Models I Biology 162 Computational Genetics Todd Vision 14 Sep 2004.
. Hidden Markov Model Lecture #6 Background Readings: Chapters 3.1, 3.2 in the text book, Biological Sequence Analysis, Durbin et al., 2001.
. Computational Genomics Lecture 8a Hidden Markov Models (HMMs) © Ydo Wexler & Dan Geiger (Technion) and by Nir Friedman (HU) Modified by Benny Chor (TAU)
1 Markov Chains Algorithms in Computational Biology Spring 2006 Slides were edited by Itai Sharon from Dan Geiger and Ydo Wexler.
Finite Mathematics & Its Applications, 10/e by Goldstein/Schneider/SiegelCopyright © 2010 Pearson Education, Inc. 1 of 60 Chapter 8 Markov Processes.
Markov Chains Chapter 16.
INDR 343 Problem Session
Stochastic Process1 Indexed collection of random variables {X t } t   for each t  T  X t is a random variable T = Index Set State Space = range.
. Class 5: Hidden Markov Models. Sequence Models u So far we examined several probabilistic model sequence models u These model, however, assumed that.
. Markov Chains Tutorial #5 © Ilan Gronau. Based on original slides of Ydo Wexler & Dan Geiger.
CS6800 Advanced Theory of Computation Fall 2012 Vinay B Gavirangaswamy
1 Introduction to Stochastic Models GSLM Outline  discrete-time Markov chain  motivation  example  transient behavior.
6. Markov Chain. State Space The state space is the set of values a random variable X can take. E.g.: integer 1 to 6 in a dice experiment, or the locations.
Entropy Rate of a Markov Chain
Decision Making in Robots and Autonomous Agents Decision Making in Robots and Autonomous Agents The Markov Decision Process (MDP) model Subramanian Ramamoorthy.
Computational Intelligence II Lecturer: Professor Pekka Toivanen Exercises: Nina Rogelj
Courtesy of J. Akinpelu, Anis Koubâa, Y. Wexler, & D. Geiger
 { X n : n =0, 1, 2,...} is a discrete time stochastic process Markov Chains.
Chapter 3 : Problems 7, 11, 14 Chapter 4 : Problems 5, 6, 14 Due date : Monday, March 15, 2004 Assignment 3.
Theory of Computations III CS-6800 |SPRING
8/14/04J. Bard and J. W. Barnes Operations Research Models and Methods Copyright All rights reserved Lecture 12 – Discrete-Time Markov Chains Topics.
1 Hidden Markov Model Observation : O1,O2,... States in time : q1, q2,... All states : s1, s2,... Si Sj.
Discrete Time Markov Chains
COMS Network Theory Week 5: October 6, 2010 Dragomir R. Radev Wednesdays, 6:10-8 PM 325 Pupin Terrace Fall 2010.
11. Markov Chains (MCs) 2 Courtesy of J. Bard, L. Page, and J. Heyl.
Markov Processes What is a Markov Process?
Goldstein/Schnieder/Lay: Finite Math & Its Applications, 9e 1 of 60 Chapter 8 Markov Processes.
Business Modeling Lecturer: Ing. Martina Hanová, PhD.
Visual Recognition Tutorial1 Markov models Hidden Markov models Forward/Backward algorithm Viterbi algorithm Baum-Welch estimation algorithm Hidden.
From DeGroot & Schervish. Example Occupied Telephone Lines Suppose that a certain business office has five telephone lines and that any number of these.
Let E denote some event. Define a random variable X by Computing probabilities by conditioning.
Discrete-time markov chain (continuation)
Markov Chains Tutorial #5
Discrete Time Markov Chains
IENG 362 Markov Chains.
Discrete-time markov chain (continuation)
Courtesy of J. Akinpelu, Anis Koubâa, Y. Wexler, & D. Geiger
Hidden Markov Model LR Rabiner
Markov Chains Tutorial #5
Presentation transcript:

. Computational Genomics Lecture 7c Hidden Markov Models (HMMs) © Ydo Wexler & Dan Geiger (Technion) and by Nir Friedman (HU) Modified by Benny Chor (TAU)

2 Outline u Finite, or Discrete, Markov Models u Hidden Markov Models u Three major questions: u Q1: Compute the probability of a given sequence of observations. A1: Forward – Backward dynamic programming algorithm (Baum Welch). u Q2: Compute the most probable sequence of states, given a sequence of observations. A2: Viterbi’s dynamic programming Algorithm u Q3: Learn best model, given an observation,. A3: The Expectation Maximization (EM) heuristic.

3 Markov Models u A discrete (finite) system: l N distinct states. l Begins (at time t=1) in some initial state(s). l At each time step (t=1,2,…) the system moves from current to next state (possibly the same as the current state) according to transition probabilities associated with current state. u This kind of system is called a finite, or discrete Markov model. Aka probabilistic finite automata. u After Andrei Andreyevich Markov ( )

4 Outline u Markov Chains (Markov Models) u Hidden Markov Chains (HMMs) u Algorithmic Questions u Biological Relevance

5 Discrete Markov Model: Example u Discrete Markov Model with 5 states. u Each a ij represents the probability of moving from state i to state j u The a ij are given in a matrix A = {a ij }  The probability to start in a given state i is  i, The vector  repre- sents these start probabilities.

6 Markov Property Markov Property: The state of the system at time t+1 depends only on the state of the system at time t X t=1 X t= 2 X t=3 X t= 4 X t=5

7 Markov Chains Stationarity In general, a process is called stationary if transition probabilities are independent of t, namely This means that if system is in state i, the probability that the system will next move to state j is p ij, no matter what the value of t is. This property clearly holds for our Markov models.

8 raining today rain tomorrow p rr = 0.4 raining today no rain tomorrow p rn = 0.6 not raining today rain tomorrow p nr = 0.2 not raining today no rain tomorrow p rr = 0.8 Simple Minded Weather Example Pr( Dry Wed. Dec. 14 Rainy Fri. Dec. 16)???

9 Simple Minded Weather Example Transition matrix for our example Note that rows sum to 1 (but columns don’t) Such matrix is called a Stochastic Matrix If the rows of a matrix and the columns of a matrix all sum to 1, we have a Doubly Stochastic Matrix

10 Coke vs. Pepsi (a cental cultural dilemma) Given that a person’s last cola purchase was Coke ™, there is a 90% chance that her next cola purchase will also be Coke ™. If that person’s last cola purchase was Pepsi™, there is an 80% chance that her next cola purchase will also be Pepsi™. coke pepsi

11 Coke vs. Pepsi Given that a person is currently a Pepsi purchaser, what is the probability that she will purchase Coke two purchases from now? The transition matrices are: (corresponding to one purchase ahead)

12 Coke vs. Pepsi Given that a person is currently a Coke drinker, what is the probability that she will purchase Pepsi three purchases from now?

13 Coke vs. Pepsi Assume each person makes one cola purchase per week. Suppose 60% of all people now drink Coke, and 40% drink Pepsi. What fraction of people will be drinking Coke three weeks from now? Let (Q 0,Q 1 )=(0.6,0.4) be the initial probabilities. We will regard Coke as 0 and Pepsi as 1 We want to find P(X 3 =0) P 00

14 Equilibrium (Stationary) Distribution coke pepsi u Suppose 60% of all people now drink Coke, and 40% drink Pepsi. What fraction will be drinking Coke 10,100,1000,10000 … weeks from now? u For each week, probability is well defined. But does it converge to some equilibrium distribution [p 0,p 1 ]? u If it does, then eqs. :.9p p 1 = p 0,.8p p 0 = p 1 must hold, yielding p 0 = 2/3, p 1 =1/3.

15 Equilibrium (Stationary) Distribution Whether or not there is a stationary distribution, and whether or not it is unique if it does exist, are determined by certain properties of the process. Irreducible means that every state is accessible from every other state. Aperiodic means that there exists at least one state for which the transition from that state to itself is possible. Positive recurrent means that the expected return time is finite for every state. coke pepsi

16 Equilibrium (Stationary) Distribution u If the Markov chain is positive recurrent, there exists a stationary distribution. If it is positive recurrent and irreducible, there exists a unique stationary distribution, and furthermore the process constructed by taking the stationary distribution as the initial distribution is ergodic (defined shortly). Then the average of a function f over samples of the Markov chain is equal to the average with respect to the stationary distribution,ergodic

17 Equilibrium (Stationary) Distribution u Writing P for the transition matrix, a stationary distribution is a vector π which satisfies the equation l Pπ = π. u In this case, the stationary distribution π is an eigenvector of the transition matrix, associated with the eigenvalue 1. eigenvectoreigenvalue

18 Discrete Markov Model - Example u States – Rainy:1, Cloudy:2, Sunny:3 u Transition matrix A – u Problem – given that the weather on day 1 (t=1) is sunny(3), what is the probability for the observation O:

19 Discrete Markov Model – Example (cont.) u The answer is -

20 Ergodicity u Ergodic model: Strongly connected - directed path w/ positive probabilities from each state i to state j (but not necessarily a complete directed graph)

21 Third Example: The Friendly Gambler Game starts with 10$ in gambler’s pocket – At each round we have the following: Gambler wins 1$ with probability p Gambler loses 1$ with probability 1-p – Game ends when gambler goes broke (no sister in bank), or accumulates a capital of 100$ (including initial capital) – Both 0$ and 100$ are absorbing states (or boundaries) 01 2 N-1 N p p p p 1-p Start (10$) or

22 Third Example: The Friendly Gambler 01 2 N-1 N p p p p 1-p Start (10$) Irreducible means that every state is accessible from every other state. Aperiodic means that there exists at least one state for which the transition from that state to itself is possible. Positive recurrent means that for every state, the expected return time is finite. If the Markov chain is positive recurrent, there exists a stationary distribution. Is the gambler’s chain positive recurrent? Does it have a stationary distribution (independent of initial distribution)?