CS723 - Probability and Stochastic Processes

Lecture No. 31

In Previous Lectures Finished discussion of random vectors and their typical applications Looked at a particular transformation of random vectors Z = X1 + X2 + X3 + … + Xn Use of convolution integral to find the PDF of sum of independent random variables Characteristic function and its application to sum of independent random variables Similarity between sum of independent random variables and a Gaussian random variable

Sequences of Numbers A given infinite sequence of real or complex numbers x1,x2,x3,…, may converge to a value x0 if lim xn = x0 as n→∞ The numbers in the infinite sequence could predictable or could be random Examples of interesting sequences of numbers xn = n = 1, 2, 3, 4 ,5, … xn = 1/n = 1, 1/2, 1/3, 1/4, … xn = (xn-1)2 with x1 > 1 or x1 < 1 xn = sqrt(xn-1) xn = cos(xn-1)

Convergence of Sequence A sequence of numbers x1,x2,x3,…, converges to a value x0 if for any selected value of δ, there is a number N s.t. for every n > N |xn – x0| < δ Example of xn = 1/n

Convergence of Sequence

Convergence of Sequence

Sequences of Functions A given infinite sequence of functions f1,f2,f3,…, which is defined on a common domain, may converge to a function f0 if, for every point xs belonging to the domain, we have lim fn(xs) = f0(xs) as n→∞ Examples of interesting sequences of functions fn(x) = xn for x ε (-1,1) or ε [-10,10] fn(x) = sin(nx) fn(x) = cos(ωx + 1/n) fn(x) = e-|nx| for all values of x ε R fn(x) = e-nx if x > 0 and 1 otherwise

Convergence of Function A sequence of functions f1,f2,f3,…, point wise converges to a function f0 , if for any point xs in the domain, the sequence f1(xs), f2(xs), f3(xs), … converges. Example: fn = e-(x-1/n) u(x)

Convergence Alm. E.W. A sequence of functions f1,f2,f3,…, converges to a function f0 almost everywhere , if length of set of points where it does not converge point wise is zero Example fn(x) = e-nx if x > 0 and 1 otherwise

Convergence of RV’s If we have an infinite collection of random variables X1, X2, X3, … defined on the same sample space, they may converge to X0 under different criteria If ξ is an outcome of the random experiments ,Xn(ξ) forms a sequence of numbers If Xn(ξ) converges to X0(ξ) for every outcome in the sample space, this convergence is called Sure Convergence The random variables cannot be in depended of each other

Convergence of RV’s Sure convergence is similar to the first type of convergence of functions with ξ taking the role of xs f the sequence of random variables does not converge for some outcomes belonging to event A, with Pr(A) = 0, then the convergence is called almost sure Almost sure convergence is also called convergence with probability 1 Similar to almost everywhere convergence of sequence of functions

Convergence of RV’s Sequences of random variables may converge under weaker types of convergences A sequence of random variables X1, X2, X3, … converges to X0 in probability if for every ε > 0 Pr(|Xn – X0| > ε) → 0 Convergence in probability and convergence with probability 1 (almost surely) are two different types of convergences Convergence with probability 1 implies convergence in probability but converse is not true

Convergence of RV’s sequence of random variables X1, X2, X3, … converges to X0 in mean square if E[|Xn – X0|2 ] → 0 Convergence in probability implies mean square convergence A sequence of random variables X1, X2, X3, … with family of CDF’s F1(x), F2(x), F3(x), … converges in distribution to a random variable X0 if Fn(x)→ F0(x) Convergence in distribution could be true for mutually independent random variables

Population Sampling Estimation of parameters of whole population from measurements on a smaller sample Mean height of students of a university from the average value of a smaller subset Mean popularity level of a president from the average values obtained from a survey In all cases, if random variable modeling the population is X, we are estimating μX from WN = (X1 + X2 + X3 + … + XN)/N This estimate will be prone to some error that is a function of size of the sample