Applied Discrete Mathematics Week 11: Relations Expected Values E(X) = 2(1/36) + 3(1/18) + 4(1/12) + 5(1/9) + 6(5/36) + 7(1/6) + 8(5/36) + 9(1/9) + 10(1/12) + 11(1/18) + 12(1/36) E(X) = 7 This means that if we roll the dice many times, sum all the numbers that appear and divide the sum by the number of trials, we expect to find a value of 7. April 4, 2017 Applied Discrete Mathematics Week 11: Relations

Applied Discrete Mathematics Week 11: Relations Expected Values Theorem: If X and Y are random variables on a sample space S, then E(X + Y) = E(X) + E(Y). Furthermore, if Xi, i = 1, 2, …, n with a positive integer n, are random variables on S, then E(X1 + X2 + … + Xn) = E(X1) + E(X2) + … + E(Xn). Moreover, if a and b are real numbers, then E(aX + b) = aE(X) + b. Note the typo in the textbook! April 4, 2017 Applied Discrete Mathematics Week 11: Relations

Applied Discrete Mathematics Week 11: Relations Expected Values Knowing this theorem, we could now solve the previous example much more easily: Let X1 and X2 be the numbers appearing on the first and the second die, respectively. For each die, there is an equal probability for each of the six numbers to appear. Therefore, E(X1) = E(X2) = (1 + 2 + 3 + 4 + 5 + 6)/6 = 7/2. We now know that E(X1 + X2) = E(X1) + E(X2) = 7. April 4, 2017 Applied Discrete Mathematics Week 11: Relations

Applied Discrete Mathematics Week 11: Relations Expected Values We can use our knowledge about expected values to compute the average-case complexity of an algorithm. Let the sample space be the set of all possible inputs a1, a2, …, an, and the random variable X assign to each aj the number of operations that the algorithm executes for that input. For each input aj, the probability that this input occurs is given by p(aj). The algorithm’s average-case complexity then is: E(X) = j=1,…,np(aj)X(aj) April 4, 2017 Applied Discrete Mathematics Week 11: Relations

Applied Discrete Mathematics Week 11: Relations Expected Values However, in order to conduct such an average-case analysis, you would need to find out: the number of steps that the algorithms takes for any (!) possible input, and the probability for each of these inputs to occur. For most algorithms, this would be a highly complex task, so we will stick with the worst-case analysis. In the textbook (page 283 in the 4th Edition, page 384 in the 5th Edition , page 431 in the 6th Edition , page 482 in the 7th Edition), an average-case analysis of the linear search algorithm is shown. April 4, 2017 Applied Discrete Mathematics Week 11: Relations

Independent Random Variables Definition: The random variables X and Y on a sample space S are independent if p(X(s) = r1  Y(s) = r2) = p(X(s) = r1)  p(Y(s) = r2). In other words, X and Y are independent if the probability that X(s) = r1  Y(s) = r2 equals the product of the probability that X(s) = r1 and the probability that Y(s) = r2 for all real numbers r1 and r2. April 4, 2017 Applied Discrete Mathematics Week 11: Relations

Independent Random Variables Example: Are the random variables X1 and X2 from the “pair of dice” example independent? Solution: p(X1 = i) = 1/6 p(X2 = j) = 1/6 p(X1 = i  X2 = j) = 1/36 Since p(X1 = i  X2 = j) = p(X1 = i)p(X2 = j) , the random variables X1 and X2 are independent. April 4, 2017 Applied Discrete Mathematics Week 11: Relations

Independent Random Variables Theorem: If X and Y are independent random variables on a sample space S, then E(XY) = E(X)E(Y). Note: E(X + Y) = E(X) + E(Y) is true for any X and Y, but E(XY) = E(X)E(Y) needs X and Y to be independent. How come? April 4, 2017 Applied Discrete Mathematics Week 11: Relations

Independent Random Variables Example: Let X and Y be random variables on some sample space, and each of them assumes the values 1 and 3 with equal probability. Then E(X) = E(Y) = 2 If X and Y are independent, we have: E(X + Y) = 1/4·(1 + 1) + 1/4·(1 + 3) + 1/4·(3 + 1) + 1/4·(3 + 3) = 4 = E(X) + E(Y) E(XY) = 1/4·(1·1) + 1/4·(1·3) + 1/4·(3·1) + 1/4·(3·3) = 4 = E(X)·E(Y) April 4, 2017 Applied Discrete Mathematics Week 11: Relations

Independent Random Variables Let us now assume that X and Y are correlated in such a way that Y = 1 whenever X = 1, and Y = 3 whenever X = 3. E(X + Y) = 1/2·(1 + 1) + 1/2·(3 + 3) = 4 = E(X) + E(Y) E(XY) = 1/2·(1·1) + 1/2·(3·3) = 5  E(X)·E(Y) April 4, 2017 Applied Discrete Mathematics Week 11: Relations

Applied Discrete Mathematics Week 11: Relations Variance The expected value of a random variable is an important parameter for the description of a random distribution. It does not tell us, however, anything about how widely distributed the values are. This can be described by the variance of a random distribution. April 4, 2017 Applied Discrete Mathematics Week 11: Relations

Applied Discrete Mathematics Week 11: Relations Variance Definition: Let X be a random variable on a sample space S. The variance of X, denoted by V(X), is V(X) = sS(X(s) – E(X))2p(s). The standard deviation of X, denoted by (X), is defined to be the square root of V(X). April 4, 2017 Applied Discrete Mathematics Week 11: Relations

Applied Discrete Mathematics Week 11: Relations Variance Useful rules: If X is a random variable on a sample space S, then V(X) = E(X2) – E(X)2. If X and Y are two independent random variables on a sample space S, then V(X + Y) = V(X) + V(Y). Furthermore, if Xi, i = 1, 2, …, n, with a positive integer n, are pairwise independent random variables on S, then V(X1 + X2 + … + Xn) = V(X1) + V(X2) + … + V(Xn). Proofs in the textbook on pages 280 and 281 (4th Edition), pages 389 and 390 (5th Edition), pages 436 and 437 (6th Edition), and pages 488-490 (7th Edition). April 4, 2017 Applied Discrete Mathematics Week 11: Relations

Applied Discrete Mathematics Week 11: Relations Now it’s Time for… Advanced Counting Techniques April 4, 2017 Applied Discrete Mathematics Week 11: Relations

Applied Discrete Mathematics Week 11: Relations Recurrence Relations A recurrence relation for the sequence {an} is an equation that expresses an is terms of one or more of the previous terms of the sequence, namely, a0, a1, …, an-1, for all integers n with n  n0, where n0 is a nonnegative integer. A sequence is called a solution of a recurrence relation if its terms satisfy the recurrence relation. April 4, 2017 Applied Discrete Mathematics Week 11: Relations

Applied Discrete Mathematics Week 11: Relations Recurrence Relations In other words, a recurrence relation is like a recursively defined sequence, but without specifying any initial values (initial conditions). Therefore, the same recurrence relation can have (and usually has) multiple solutions. If both the initial conditions and the recurrence relation are specified, then the sequence is uniquely determined. April 4, 2017 Applied Discrete Mathematics Week 11: Relations

Applied Discrete Mathematics Week 11: Relations Recurrence Relations Example: Consider the recurrence relation an = 2an-1 – an-2 for n = 2, 3, 4, … Is the sequence {an} with an=3n a solution of this recurrence relation? For n  2 we see that 2an-1 – an-2 = 2(3(n – 1)) – 3(n – 2) = 3n = an. Therefore, {an} with an=3n is a solution of the recurrence relation. April 4, 2017 Applied Discrete Mathematics Week 11: Relations

Applied Discrete Mathematics Week 11: Relations Recurrence Relations Is the sequence {an} with an=5 a solution of the same recurrence relation? For n  2 we see that 2an-1 – an-2 = 25 - 5 = 5 = an. Therefore, {an} with an=5 is also a solution of the recurrence relation. April 4, 2017 Applied Discrete Mathematics Week 11: Relations

Modeling with Recurrence Relations Example: Someone deposits $10,000 in a savings account at a bank yielding 5% per year with interest compounded annually. How much money will be in the account after 30 years? Solution: Let Pn denote the amount in the account after n years. How can we determine Pn on the basis of Pn-1? April 4, 2017 Applied Discrete Mathematics Week 11: Relations

Modeling with Recurrence Relations We can derive the following recurrence relation: Pn = Pn-1 + 0.05Pn-1 = 1.05Pn-1. The initial condition is P0 = 10,000. Then we have: P1 = 1.05P0 P2 = 1.05P1 = (1.05)2P0 P3 = 1.05P2 = (1.05)3P0 … Pn = 1.05Pn-1 = (1.05)nP0 We now have a formula to calculate Pn for any natural number n and can avoid the iteration. April 4, 2017 Applied Discrete Mathematics Week 11: Relations

Modeling with Recurrence Relations Let us use this formula to find P30 under the initial condition P0 = 10,000: P30 = (1.05)3010,000 = 43,219.42 After 30 years, the account contains $43,219.42. April 4, 2017 Applied Discrete Mathematics Week 11: Relations

Modeling with Recurrence Relations Another example: Let an denote the number of bit strings of length n that do not have two consecutive 0s (“valid strings”). Find a recurrence relation and give initial conditions for the sequence {an}. Solution: Idea: The number of valid strings equals the number of valid strings ending with a 0 plus the number of valid strings ending with a 1. April 4, 2017 Applied Discrete Mathematics Week 11: Relations