Probability theory 2011 Convergence concepts in probability theory  Definitions and relations between convergence concepts  Sufficient conditions for almost sure convergence  Convergence via transforms  The law of large numbers and the central limit theorem

Probability theory 2011 Coin-tossing: relative frequency of heads Convergence of each trajectory? Convergence in probability?

Probability theory 2011 Convergence to a constant The sequence {X n } of random variables converges almost surely to the constant c if and only if P({  ; X n (  )  c as n   }) = 1 The sequence {X n } of random variables converges in probability to the constant c if and only if, for all  > 0, P({  ; | X n (  ) – c| >  })  0 as n  

Probability theory 2011 An (artificial) example Let X 1, X 2, … be a sequence of independent binary random variables such that P(X n = 1) = 1/n and P(X n = 0) = 1 – 1/n Does X n converge to 0 in probability? Does X n converge to 0 almost surely? Common exception set?

Probability theory 2011 The law of large numbers for random variables with finite variance Let {X n } be a sequence of independent and identically distributed random variables with mean  and variance  2, and set S n = X 1 + … + X n Then Proof: Assume that  = 0. Then.

Probability theory 2011 Convergence to a random variable: definitions The sequence {X n } of random variables converges almost surely to the random variable X if and only if P({  ; X n (  )  X(  ) as n   }) = 1 Notation: The sequence {X n } of random variables converges in probability to the random variable X if and only if, for all  > 0, P({  ; | X n (  ) – X(  )| >  })  0 as n   Notation:

Probability theory 2011 Convergence to a random variable: an example Assume that the concentration of NO in air is continuously recorded and let X t, be the concentration at time t. Consider the random variables: Does Y n converge to Y in probability? Does Y n converge to Y almost surely?

Probability theory 2011 Convergence in distribution: an example Let X n  Bin(n, c/n). Then the distribution of X n converges to a Po(c) distribution as n  . p = 0.1)

Probability theory 2011 Convergence in distribution and in norm The sequence X n converges in distribution to the random variable X as n   iff for all x where F X (x) is continuous. Notation: The sequence X n converges in quadratic mean to the random variable X as n   iff Notation:

Probability theory 2011 Relations between the convergence concepts Almost sure convergence Convergence in r-mean Convergence in probability Convergence in distribution

Probability theory 2011 Convergence in probability implies convergence in distribution Note that, for all  > 0,

Probability theory 2011 Convergence almost surely - convergence in r-mean Consider a branching process in which the offspring distribution has mean 1. Does it converge to zero almost surely? Does it converge to zero in quadratic mean? Let X 1, X 2, … be a sequence of independent random variables such that P(X n = n 2 ) = 1/n 2 and P(X n = 0) = 1 – 1/n 2 Does X n converge to 0 in probability? Does X n converge to 0 almost surely? Does X n converge to 0 in quadratic mean?

Probability theory 2011 Relations between different types of convergence to a constant Almost sure convergence Convergence in r-mean Convergence in probability Convergence in distribution

Probability theory 2011 Convergence via generating functions Let X, X 1, X 2, … be a sequence of nonnegative, integer- valued random variables, and suppose that Then Is the limit function of a sequence of generating functions a generating function?

Probability theory Convergence via moment generating functions Let X, X 1, X 2, … be a sequence of random variables, and suppose that Then Is the limit function of a sequence of moment generating functions a moment generating function?

Probability theory 2011 Convergence via characteristic functions Let X, X 1, X 2, … be a sequence of random variables, and suppose that Then Is the limit function of a sequence of characteristic functions a characteristic function?

Probability theory 2011 Convergence to a constant via characteristic functions Let X 1, X 2, … be a sequence of random variables, and suppose that Then

Probability theory 2011 The law of large numbers (for variables with finite expectation) Let {X n } be a sequence of independent and identically distributed random variables with expectation , and set S n = X 1 + … + X n Then.

Probability theory 2011 The strong law of large numbers (for variables with finite expectation) Let {X n } be a sequence of independent and identically distributed random variables with expectation , and set S n = X 1 + … + X n Then.

Probability theory 2011 The central limit theorem Let {X n } be a sequence of independent and identically distributed random variables with mean  and variance  2, and set S n = X 1 + … + X n Then Proof: If  = 0, we get.

Probability theory 2011 Rate of convergence in the central limit theorem Example: X  U(0,1).

Probability theory 2011 Sums of exponentially distributed random variables

Probability theory 2011 Convergence of empirical distribution functions Proof: Write F n (x) as a sum of indicator functions Bootstrap techniques: The original distribution is replaced with the empirical distribution

Probability theory 2011 Resampling techniques - the bootstrap method Sampling with replacement Resampled data Observed data

Probability theory 2011 Characteristics of infinite sequences of events Let {A n, n = 1, 2, …} be a sequence of events, and define Example: Consider a queueing system and let A n = {the queueing system is empty at time n }

Probability theory 2011 The probability that an event occurs infinitely often - Borel-Cantelli’s first lemma Let {A n, n = 1, 2, …} be an arbitrary sequence of events. Then Example: Consider a queueing system and let A n = {the queueing system is empty at time n } Is the converse true?

Probability theory 2011 The probability that an event occurs infinitely often - Borel-Cantelli’s second lemma Let {A n, n = 1, 2, …} be a sequence of independent events. Then

Probability theory 2011 Necessary and sufficient conditions for almost sure convergence of independent random variables Let X 1, X 2, … be a sequence of independent random variables. Then

Probability theory 2011 Exercises: Chapter VI 6.1, 6.6, 6.9, 6.10, 6.17, 6.21, 6.25, 6.49