1. Probability theory 2011 Convergence concepts in probability theory  Definitions and relations between convergence concepts  Sufficient conditions for almost sure convergence  Convergence via transforms  The law of large numbers and the central limit theorem

2. Probability theory 2011 Coin-tossing: relative frequency of heads Convergence of each trajectory? Convergence in probability?

3. Probability theory 2011 Convergence to a constant The sequence {X n } of random variables converges almost surely to the constant c if and only if P({  ; X n (  )  c as n   }) = 1 The sequence {X n } of random variables converges in probability to the constant c if and only if, for all  > 0, P({  ; | X n (  ) – c| >  })  0 as n  

4. Probability theory 2011 An (artificial) example Let X 1, X 2, … be a sequence of independent binary random variables such that P(X n = 1) = 1/n and P(X n = 0) = 1 – 1/n Does X n converge to 0 in probability? Does X n converge to 0 almost surely? Common exception set?

5. Probability theory 2011 The law of large numbers for random variables with finite variance Let {X n } be a sequence of independent and identically distributed random variables with mean  and variance  2, and set S n = X 1 + … + X n Then Proof: Assume that  = 0. Then.

6. Probability theory 2011 Convergence to a random variable: definitions The sequence {X n } of random variables converges almost surely to the random variable X if and only if P({  ; X n (  )  X(  ) as n   }) = 1 Notation: The sequence {X n } of random variables converges in probability to the random variable X if and only if, for all  > 0, P({  ; | X n (  ) – X(  )| >  })  0 as n   Notation:

7. Probability theory 2011 Convergence to a random variable: an example Assume that the concentration of NO in air is continuously recorded and let X t, be the concentration at time t. Consider the random variables: Does Y n converge to Y in probability? Does Y n converge to Y almost surely?

8. Probability theory 2011 Convergence in distribution: an example Let X n  Bin(n, c/n). Then the distribution of X n converges to a Po(c) distribution as n  . p = 0.1)

9. Probability theory 2011 Convergence in distribution and in norm The sequence X n converges in distribution to the random variable X as n   iff for all x where F X (x) is continuous. Notation: The sequence X n converges in quadratic mean to the random variable X as n   iff Notation:

10. Probability theory 2011 Relations between the convergence concepts Almost sure convergence Convergence in r-mean Convergence in probability Convergence in distribution

11. Probability theory 2011 Convergence in probability implies convergence in distribution Note that, for all  > 0,

12. Probability theory 2011 Convergence almost surely - convergence in r-mean Consider a branching process in which the offspring distribution has mean 1. Does it converge to zero almost surely? Does it converge to zero in quadratic mean? Let X 1, X 2, … be a sequence of independent random variables such that P(X n = n 2 ) = 1/n 2 and P(X n = 0) = 1 – 1/n 2 Does X n converge to 0 in probability? Does X n converge to 0 almost surely? Does X n converge to 0 in quadratic mean?

13. Probability theory 2011 Relations between different types of convergence to a constant Almost sure convergence Convergence in r-mean Convergence in probability Convergence in distribution

14. Probability theory 2011 Convergence via generating functions Let X, X 1, X 2, … be a sequence of nonnegative, integer- valued random variables, and suppose that Then Is the limit function of a sequence of generating functions a generating function?

15. Probability theory 20101 Convergence via moment generating functions Let X, X 1, X 2, … be a sequence of random variables, and suppose that Then Is the limit function of a sequence of moment generating functions a moment generating function?

16. Probability theory 2011 Convergence via characteristic functions Let X, X 1, X 2, … be a sequence of random variables, and suppose that Then Is the limit function of a sequence of characteristic functions a characteristic function?

17. Probability theory 2011 Convergence to a constant via characteristic functions Let X 1, X 2, … be a sequence of random variables, and suppose that Then

18. Probability theory 2011 The law of large numbers (for variables with finite expectation) Let {X n } be a sequence of independent and identically distributed random variables with expectation , and set S n = X 1 + … + X n Then.

19. Probability theory 2011 The strong law of large numbers (for variables with finite expectation) Let {X n } be a sequence of independent and identically distributed random variables with expectation , and set S n = X 1 + … + X n Then.

20. Probability theory 2011 The central limit theorem Let {X n } be a sequence of independent and identically distributed random variables with mean  and variance  2, and set S n = X 1 + … + X n Then Proof: If  = 0, we get.

21. Probability theory 2011 Rate of convergence in the central limit theorem Example: X  U(0,1).

22. Probability theory 2011 Sums of exponentially distributed random variables

23. Probability theory 2011 Convergence of empirical distribution functions Proof: Write F n (x) as a sum of indicator functions Bootstrap techniques: The original distribution is replaced with the empirical distribution

24. Probability theory 2011 Resampling techniques - the bootstrap method 34 67 7988 39 41 85 70 62 905844 60 73 22 58 7988 41 88 85 70 90 223444 60 41 60 Sampling with replacement Resampled data Observed data

25. Probability theory 2011 Characteristics of infinite sequences of events Let {A n, n = 1, 2, …} be a sequence of events, and define Example: Consider a queueing system and let A n = {the queueing system is empty at time n }

26. Probability theory 2011 The probability that an event occurs infinitely often - Borel-Cantelli’s first lemma Let {A n, n = 1, 2, …} be an arbitrary sequence of events. Then Example: Consider a queueing system and let A n = {the queueing system is empty at time n } Is the converse true?

27. Probability theory 2011 The probability that an event occurs infinitely often - Borel-Cantelli’s second lemma Let {A n, n = 1, 2, …} be a sequence of independent events. Then

28. Probability theory 2011 Necessary and sufficient conditions for almost sure convergence of independent random variables Let X 1, X 2, … be a sequence of independent random variables. Then

29. Probability theory 2011 Exercises: Chapter VI 6.1, 6.6, 6.9, 6.10, 6.17, 6.21, 6.25, 6.49