1. CS723 - Probability and Stochastic Processes

2. Lecture No. 31

3. In Previous Lectures
Finished discussion of random vectors and their typical applications
Looked at a particular transformation of random vectors Z = X1 + X2 + X3 + … + Xn
Use of convolution integral to find the PDF of sum of independent random variables
Characteristic function and its application to sum of independent random variables
Similarity between sum of independent random variables and a Gaussian random variable

4. Sequences of Numbers
A given infinite sequence of real or complex numbers x1,x2,x3,…, may converge to a value x0 if lim xn = x0 as n→∞ The numbers in the infinite sequence could predictable or could be random Examples of interesting sequences of numbers xn = n = 1, 2, 3, 4 ,5, … xn = 1/n = 1, 1/2, 1/3, 1/4, … xn = (xn-1)2 with x1 > 1 or x1 < 1 xn = sqrt(xn-1) xn = cos(xn-1)

5. Convergence of Sequence
A sequence of numbers x1,x2,x3,…, converges to a value x0 if for any selected value of δ, there is a number N s.t. for every n > N |xn – x0| < δ Example of xn = 1/n

6. Convergence of Sequence

7. Convergence of Sequence

8. Sequences of Functions
A given infinite sequence of functions f1,f2,f3,…, which is defined on a common domain, may converge to a function f0 if, for every point xs belonging to the domain, we have lim fn(xs) = f0(xs) as n→∞ Examples of interesting sequences of functions fn(x) = xn for x ε (-1,1) or ε [-10,10] fn(x) = sin(nx) fn(x) = cos(ωx + 1/n) fn(x) = e-|nx| for all values of x ε R fn(x) = e-nx if x > 0 and 1 otherwise

9. Convergence of Function
A sequence of functions f1,f2,f3,…, point wise converges to a function f0 , if for any point xs in the domain, the sequence f1(xs), f2(xs), f3(xs), … converges. Example: fn = e-(x-1/n) u(x)

10. Convergence Alm. E.W.
A sequence of functions f1,f2,f3,…, converges to a function f0 almost everywhere , if length of set of points where it does not converge point wise is zero Example fn(x) = e-nx if x > 0 and 1 otherwise

11. Convergence of RV’s
If we have an infinite collection of random variables X1, X2, X3, … defined on the same sample space, they may converge to X0 under different criteria
If ξ is an outcome of the random experiments ,Xn(ξ) forms a sequence of numbers
If Xn(ξ) converges to X0(ξ) for every outcome in the sample space, this convergence is called Sure Convergence
The random variables cannot be in depended of each other

12. Convergence of RV’s
Sure convergence is similar to the first type of convergence of functions with ξ taking the role of xs
f the sequence of random variables does not converge for some outcomes belonging to event A, with Pr(A) = 0, then the convergence is called almost sure
Almost sure convergence is also called convergence with probability 1
Similar to almost everywhere convergence of sequence of functions

13. Convergence of RV’s
Sequences of random variables may converge under weaker types of convergences
A sequence of random variables X1, X2, X3, … converges to X0 in probability if for every ε > 0 Pr(|Xn – X0| > ε) → 0
Convergence in probability and convergence with probability 1 (almost surely) are two different types of convergences
Convergence with probability 1 implies convergence in probability but converse is not true

14. Convergence of RV’s
sequence of random variables X1, X2, X3, … converges to X0 in mean square if E[|Xn – X0|2 ] → 0
Convergence in probability implies mean square convergence
A sequence of random variables X1, X2, X3, …
with family of CDF’s F1(x), F2(x), F3(x), … converges in distribution to a random variable X0 if Fn(x)→ F0(x)
Convergence in distribution could be true for mutually independent random variables

15. Population Sampling
Estimation of parameters of whole population from measurements on a smaller sample
Mean height of students of a university from the average value of a smaller subset
Mean popularity level of a president from the average values obtained from a survey
In all cases, if random variable modeling the population is X, we are estimating μX from WN = (X1 + X2 + X3 + … + XN)/N
This estimate will be prone to some error that is a function of size of the sample