1. K. Shum Lecture 26 Singular random variable Strong law of large number

2. K. Shum Random variable Both discrete and continuous r.v. can be described by cumulative distribution function F(x)=P(X  x). Discrete F(x) x 1 Derivative is zero almost everywhere. Number of discontinuity points are finite or countably infinite. F(x) Continuous x 1

3. K. Shum Singular Random variable Singular -- Neither discrete nor continuous. Devil’s staircase : a continuous function rising from 0 to 1 on the unit interval whose derivative is zero almost everywhere. Closely related to the Cantor’s set. Uncountably many steps

4. K. Shum Iterative construction The Devil’s staircase can be constructed by the following algorithm. % ERG2040A Lecture26 Construction of a singular cdf x = [0 1/3 2/3 1]; y = [0 1/2 1/2 1]; for k=1:6 figure(k) plot(x,y) axis square x = [x/3 2/3+x/3]; y = [y/2 1/2+y/2]; end figure(k+1) plot(x,y) title('Cumulative distribution function of a singular random variable') axis square xlabel('x'); ylabel('F(x)')

5. K. Shum Discrete, continuous and singular Continuous: cdf is differentiable. Discrete: Derivative of cdf is zero except for finitely or countably many points. Singular: Derivative of cdf is zero almost everywhere. Number of non-differentiable points many be uncountably infinitely. A unifying framework for discrete, continuous and singular is the measure theory and Lebesgue integral (MAT4050/5011/5012.)

6. K. Shum In the game of CLi vs KKLeung, there are sample paths that are non-terminating. “Probability = 0”  Impossible …… However, the event of all such non-terminating paths has probability zero. With probability 1, somebody will win eventually.

7. K. Shum Ma Jong 4 people play Ma Jong for infinitely many times. P(E wins indefinitely)=0 –P(EEEEEEEEE……..)=0 P(S wins finitely many times)=0 P(No body wins two times consecutively)=0, –P(ESWN…, or SWNE…, or WNES…, or NESW…)=0 P(No body wins more than 10 times consecutively) =0 P(When ever W loses, N will win in the next game)=0 P(complement of the above)=1 E S W N 3/4 1/4

8. K. Shum Normal number theorem There are (in fact uncountably many) infinitely long 0-1 sequences (X 1,X 2,X 3,…) such that the limit does not exist. –e.g.*11*11*11*11*11*11*11*11…, where * is either 0 or 1) If the X i are chosen independently and equally likely, the set of all such sequences has probability zero. Indeed, with probability 1, the limit exists and equals to 0.5.

9. K. Shum Strong law Version 1 Let X 1, X 2, X 3, X 4, X 5,…, be IID (independent and identically distributed) random variables with finite mean. Let S n = X 1 + X 2 +…+ X n. With probability one, Implications: with probability zero: –the limit does not exists. –the limit exists but S n /n approaches the wrong limit.

10. K. Shum Strong law version 2 For any  >0, “sup” stands for supremum. Roughly speaking, supremum is the maximum. In words, if we plot S n /n against n, with probability 1, The graph is bounded between E[x]  eventually, for any  >0. E[X] E[X]+  E[X]– 

11. K. Shum Strong vs Weak Since both strong law and weak law hold in the case of IID random variable, we cannot illustrate their difference using IID r.v’s. Example: Let S n, n  1, be independent random variables. S n =n with probability 1/n, and 0 otherwise. –E[S n ]=1 for all n. –We can interpret S n as sum of X 1 +…+X n. Then E[X n ]=0 for all n. Weak Law: for any  >0, No supremum in the weak law

12. K. Shum Strong law fails in this example S n /n is either 0 or 1. Take  =0.5. Fix any n, the event that S m /m is within 0  0.5 for all m  n has probability which can be shown equal to zero. But weak law holds. 1 S n /n n

13. K. Shum H20H20 Close book, open notes with no page limitation. Bring calculator 6 questions. No proof, no bonus. Include material before and during midterm. P(Combinatorics)=1. P(probability Density function, cumulative distribution function) = 1 P(Mean, variance, covariance) = 1 P(Gaussian, exponential)=1 P(Central limit theorem)=1 P(Chebyshev inequality)=1 P(Markov chain)=1 P(Cauchy-Schwarz inequality)=0.5 P(Characteristic function)=0 P(Content in this file)=0