Markov Chains & Randomized algorithms BY HAITHAM FALLATAH FOR COMP4804
Outline History Of Markov chains The Markov property Markov Chains Example 3 MC models and applications Analysis of a randomized algorithm using MC
History of the Markov chain Andrey Markov was a Russian mathematician Lived in 19 th and early 20 th century He was bad at school in everything but math Student of Chebyshev was motivated to work on a model that was later known as the MC because of 2 reasons. Show that Chebyshev’s approach to extending the Law of Large numbers to sums of dependent random variables could be improved. He had an animosity with another Russian mathematician called Nekrasov who claimed that only independent events can converge on predictable distributions
The Markov property Markov disproved Nekrasov’s claim by coming up with a model that has states and transitions. Pick a ball at random, if Blue, pick a ball from Bin 1 otherwise from Bin 2 Probability will converge on a predictable distribution 1:1 Red:Blue 2:1 Red:Blue 1/3 2/3 1/2 Bin 1Bin 2 1/2
Markov Chain Example SunnyCloudyRainySnowy Sunny Cloudy Rainy Snowy SunnyCloudyRainySnowy Transition Matrix P 100 matrix for gen(100) SunnyCloudyRainySnowy 1000 P 0 matrix for initial state
Markov Models & applications Discreet time Markov chain (DTMC) It is the notion that there is a chain of steps between states where movement from a state to the next, depends only on the current state. Pr(X n+1 = x | X 1 = x 1, X 2 = x 2, ….., X n = x n ) = Pr(X n+1 = x | X n = x n ) This process is used with simulations and analysing random algorithms
Markov Models & applications Continues Time Markov chain (CTMC) Similar to the DTMC model except that steps have no meaning in continuous time so the Monrovian property must hold for all future times instead of just for one step. This model is best suited for modeling biological and physical systems because of their continues time nature. Used in construction of phylogenies (evolutionary trees)
Markov Models & applications Hidden Markov chain model It is a model in which the “real Markov chain” is hidden and instead, another set of states is observed and is dependent on the states of the hidden chain. This is one of the most powerful Markov models because it can fill in the blanks in problems where not all the information is given. This model is used in pattern, speech, and writing recognition Z1Z1 Z2Z2 Z3Z3 ZnZn X1X1 X2X2 X3X3 XnXn
Analysis a randomized algorithm One example of a random algorithm that can be analysed by the Markov chain model, is a randomized algorithm that solves the 2- SAT problem. The 2-SAT problem consists of n Boolean variables and m clauses that are in a 2-conjunctive normal form. This problem is solved when all clauses have a value of true and thus the overall result of the statement would be true. This problem is considered in P (polynomial) as many algorithms can solve it in polynomial time.
The Problem We are given an algorithm that takes O(n 2 ) steps to find a satisfiable assignment. Our job is to prove independently using the Markov chain that this claim is true. The algorithm runs as follows (pseudo code): Repeat the following 2mn 2 times Select a clause that’s not satisfied (ie. False) at random Select one of the variables in the clause at random and flip its value If the formula is satisfied, return true Otherwise, return that this formula is unsatisfiable.
Example of Algorithm Execution X 1 = True X 2 = False X 3 = False X 4 = False X 1 = False X 2 = False X 3 = False X 4 = False X 1 = True X 2 = False X 3 = True X 4 = False X 1 = True X 2 = False X 3 = False X 4 = False
Analysis
j-1 jj+1n 0 1/2
Analysis
Thus:
Analysis The System of equations we have is: Using Induction we can get:
Analysis Finally, we can substitute and get:
References Probability and Computing (book) by Michael mitzenmacher and Eli Upfal (Main) Markov Models and Hidden Markov Models: A Brief Tutorial Discrete Mathematics & Mathematical Reasoning by Kousha Etessami ut_LJSA_2.pdf Khan Academy