1 Chap 5 Sums of Random Variables and Long-Term Averages Many problems involve the counting of number of occurrences of events, computation of arithmetic averages in a series of measurements. These problems can be reduced to the problem of finding the distribution of a random variable that consists of sum of n i.i.d. random variables.

2 5.1 Sums of Random Variables

3 Ex. 5.2

4 The characteristic function of S n

5 Ex. 5.3 Sum of n iid Gaussian r.v. with parameters and. Ex. 5.5 Sum of n iid exponential r.v. with parameter p.102 n-Erlang

6 If are integer-valued r.v.s., it is preferable to use the prob. generating function (z-transform). Ex. Find the generating function for a sum of n iid geometrically distributed r.v. p.100, negative binomial

7 Sum of a random number of random variables N is a r.v., independent of Ex. 5.7

8

9 5.2 Sample Mean and Laws of Large Numbers

10 Using Chebyshev inequality

11

12 Weak Law of Larger Numbers Strong Law of Larger Numbers Fig. 5.1 Sample mean will be close to the true mean with high probability With probability 1, every sequence of sample mean calculations will eventually approach and stay close to E[X]. n MnMn

13

The central Limit Theorem then CLT: as n becomes large, cdf of S n approach that of a Gaussian.

15

16 characteristic function of a zero-mean, unit-variance Gaussian r.v. Fig show approx.

17

18

Confidence Intervals If is small, Xj’s are tightly clustered about Mn. and we can be confident that Mn is close to E[X ].

20

21 Case 1. Xj’s Gaussian with unknown Mean and known Variance M n is Gaussian with mean and variance

22 Table5.1

23 Case2: Gaussian; Mean and Variance unknown use sample variance as replacement of variance the confidence interval becomes Zero-mean unit-variance Gaussian Indep. W is a student’s t-distribution with n-1 degrees of freedom.

24 (Ex. 4.38)

25 Table 5.2

26 Case 3: non-Gaussian; Mean and Variance unknown. Use method of batch mean. Performing a series of M independent experiments in which sample mean (from a large number of observations) is computed.

Convergence of Sequences of Random Variables a sequence of functions of

28 Ex a sequence of real number for a given n

29 N n

30 Ex: Strong Law of Large numbers n

31

32

33 Mean-Square Convergence Convergence in Probability Ex: weak law of large numbers. n0 n

34 Ex. Central limit theorem Ex. 5.21: Bernoulli iid sequence

35 dist prob a.s. s m.s.