1. Chapter 5 Markov processes Run length coding Gray code

2. | transition probability Markov Processes Let S = {s 1, …, s q } be a set of symbols. A j th -order Markov process has probabilities p(s i | s i 1 … s i j ) associated with it, the conditional probability of seeing s i after seeing s i 1 … s i j. This is said to be a j- memory source, and there are q j states in the Markov process. Transition Graph b ½ ½ c ¼ ¼ a ⅓ ⅓ ⅓ ¼ ¼ sjsj sisi p(s i | s j ) Weather Example: Let (j = 1). Think: a means “fair” b means “rain” c means “snow” Transition Matrix p(s i | s j ) i = column, j = row next symbol = M 5.2 currentstatecurrentstate ∑ outgoing edges = 1 ⅓ ⅓ ⅓ ¼ ½ ¼ ¼ ¼ ½ abc a b c

3. Ergodic Equilibriums Definition: A Markov process M is said to be ergodic if 1.From any state we can eventually get to any other state. 2.The system reaches a limiting distribution. 5.2

4. Predictive Coding Assume a prediction algorithm for the source which given all prior symbols, predicts the next. s 1 ….. s n  1  p n e n = p n  s n error input stream prediction What is transmitted is the error, e i. By knowing just the error, the predictor also knows the original symbols. source  channel  destination predictor enen enen snsn snsn pnpn pnpn must assume that both predictors are identical, and start in the same state 5.7

5. Accuracy: The probability of the predictor being correct is p = 1  q; constant (over time) and independent of other prediction errors. Let the probability of a run of exactly n 0’s, (0 n 1), be p(n) = p n ∙ q. The probability of runs of any length n = 0, 1, 2, … is: Note: alternate method for calculating f(p), look at 5.8

6. Coding of Run Lengths Send a k-digit binary number to represent a run of zeroes whose length is between 0 and 2 k  2. (small runs are in binary) For run lengths larger than 2 k  2, send 2 k  1 (k ones) followed by another k-digit binary number, etc. (large runs are in unary) Let n = run length. Fix k = block length. Let n = i ∙ m + j0 ≤ j < m = 2 k  1 like “reading” the “matrix” with m cells and ∞ many rows. 5.9

7. Let p(n) = the probability of a run of exactly n 0’s: 0 n 1. The expected code length is: 5.9 But every n can be written uniquely as i∙m + j where i ≥ 0, 0 ≤ j < m = 2 k  1. Expected length of run length code

8. Gray Code Consider an analog-to-digital “flash” converter consisting of a rotating wheel: 00 0 0 0 0 0 00 0 0 00 1 1 1 1 1 1 1 1 1 11 1 The maximum error in the scheme is ± ⅛ rotation because … imagine “brushes” contacting the wheel in each of the three circles The Hamming Distance between adjacent positions is 1. In ordinary binary, the maximum distance is 3 (the max. possible). 5.15-17

9. 1-bit Gray code 5.15-17