1. STA347 - week 91 Random Vectors and Matrices A random vector is a vector whose elements are random variables. The collective behavior of a p x 1 random vector is described by a joint probability density function f(x 1,x 2,…,x p ) = f(x). If the joint density of a p x 1 random vector can be factored as f(x 1,x 2,…,x p ) = f 1 (x 1 ) f 2 (x 2 )∙∙∙ f p (x p ) then the p continuous random variables X 1,X 2,…X p are mutually independent.

2. STA347 - week 92 Mean and Variance of Random Vector The expected value of a random vector is a vector of the expected values of each of its elements. That is, the population mean vector is The population variance-covariance matrix of a px1 random vector x is a p x p symmetric matrix where σ ij = Cov(X i, X j ) = E(X i – μ i )(X j – μ j ). The population correlation matrix of a px1 random vector x is a p x p symmetric matrix ρ = (ρ ij ) where

3. STA347 - week 93 Properties of Mean Vector and Covariance Matrix

4. STA347 - week 94 Functions of Random variables In some case we would like to find the distribution of Y = h(X) when the distribution of X is known. Discrete case Examples 1. Let Y = aX + b, a ≠ 0 2. Let

5. STA347 - week 95 Continuous case – Examples 1. Suppose X ~ Uniform(0, 1). Let, then the cdf of Y can be found as follows The density of Y is then given by 2. Let X have the exponential distribution with parameter λ. Find the density for 3. Suppose X is a random variable with density Check if this is a valid density and find the density of.

6. STA347 - week 96 Theorem If X is a continuous random variable with density f X (x) and h is strictly increasing and differentiable function form R  R then Y = h(X) has density for. Proof:

7. STA347 - week 97 Theorem If X is a continuous random variable with density fX(x) and h is strictly decreasing and differentiable function form R  R then Y = h(X) has density for. Proof:

8. STA347 - week 98 Summary If Y = h(X) and h is monotone then Example X has a density Let. Compute the density of Y.

9. STA347 - week 99 Change-of-Variable for Joint Distributions Theorem Let X and Y be jointly continuous random variables with joint density function f X,Y (x,y) and let D XY = {(x,y): f X,Y (x,y) >0}. If the mapping T given by T(x,y) = (u(x,y),v(x,y)) maps D XY onto D UV. Then U, V are jointly continuous random variable with joint density function given by where J(u,v) is the Jacobian of T -1 given by assuming derivatives exists and are continuous at all points in D UV.

10. STA347 - week 910 Example Let X, Y have joint density function given by Find the density function of

11. STA347 - week 911 Example Show that the integral over the Standard Normal distribution is 1.

12. STA347 - week 912 Example A device containing two key components fails when and only when both components fail. The lifetime, T 1 and T 2, of these components are independent with a common density function given by The cost, X, of operating the device until failure is 2T 1 + T 2. Find the density function of X.

13. STA347 - week 913 Convolution Suppose X, Y jointly distributed random variables. We want to find the probability / density function of Z=X+Y. Discrete case X, Y have joint probability function p X,Y (x,y). Z = z whenever X = x and Y = z – x. So the probability that Z = z is the sum over all x of these joint probabilities. That is If X, Y independent then This is known as the convolution of p X (x) and p Y (y).

14. STA347 - week 914 Example Suppose X~ Poisson(λ 1 ) independent of Y~ Poisson(λ 2 ). Find the distribution of X+Y.

15. 15 Convolution - Continuous case Suppose X, Y random variables with joint density function f X,Y (x,y). We want to find the density function of Z=X+Y. Can find distribution function of Z and differentiate. How? The Cdf of Z can be found as follows: If is continuous at z then the density function of Z is given by If X, Y independent then This is known as the convolution of f X (x) and f Y (y).

16. STA347 - week 916 Example X, Y independent each having Exponential distribution with mean 1/λ. Find the density for W=X+Y.

17. STA347 - week 917 Order Statistics The order statistics of a set of random variables X 1, X 2,…, X n are the same random variables arranged in increasing order. Denote by X (1) = smallest of X 1, X 2,…, X n X (2) = 2 nd smallest of X 1, X 2,…, X n X (n) = largest of X 1, X 2,…, X n Note, even if X i ’s are independent, X (i) ’s can not be independent since X (1) ≤ X (2) ≤ … ≤ X (n) Distribution of X i ’s and X (i) ’s are NOT the same.

18. STA347 - week 918 Distribution of the Largest order statistic X (n) Suppose X 1, X 2,…, X n are i.i.d random variables with common distribution function F X (x) and common density function f X (x). The CDF of the largest order statistic, X (n), is given by The density function of X (n) is then

19. STA347 - week 919 Example Suppose X 1, X 2,…, X n are i.i.d Uniform(0,1) random variables. Find the density function of X (n).

20. STA347 - week 920 Distribution of the Smallest order statistic X (1) Suppose X 1, X 2,…, X n are i.i.d random variables with common distribution function F X (x) and common density function f X (x). The CDF of the smallest order statistic X (1) is given by The density function of X (1) is then

21. STA347 - week 921 Example Suppose X 1, X 2,…, X n are i.i.d Uniform(0,1) random variables. Find the density function of X (1).

22. STA347 - week 922 Distribution of the kth order statistic X (k) Suppose X 1, X 2,…, X n are i.i.d random variables with common distribution function F X (x) and common density function f X (x). The density function of X (k) is

23. STA347 - week 923 Example Suppose X 1, X 2,…, X n are i.i.d Uniform(0,1) random variables. Find the density function of X (k).

24. STA347 - week 924 Computer Simulations - Introduction Modern high-speed computers can be used to perform simulation studies. Computer simulation methods are commonly used in statistical applications; sometimes they replace theory, e.g., bootstrap methods. Computer simulations are becoming more and more common in many applications such as quality control, marketing, scientific research etc.

25. STA347 - week 925 Applications of Computer Simulations Our main focus is on probabilistic simulations. Examples of applications of such simulations include:  Simulate probabilities and random variables numerically.  Approximate quantities that are too difficult to compute mathematically.  Random selection of a sample from a very large data sets.  Encrypt data or generate passwords.  Generate potential solutions for difficult problems.

26. STA347 - week 926 Steps in Probabilistic Simulations In most applications, the first step is to specify a certain probability distribution. Once such distribution is specified, it will be desired to generate one or more random variables having that distribution. The build-in computer device that generates random numbers is called pseudorandom number generator. It is a device for generating a sequence U 1, U 2, … of random values that are approximately independent and have approximately uniform distribution of the unit interval [0,1].

27. STA347 - week 927 Simulating Discrete Distributions - Example Suppose we wish to generate X ~ Bernoulli(p), where 0 < p < 1. We start by generating U ~ Uniform[0, 1] and then set: Then clearly X takes two values, 0 and 1. Further, Therefore, we have that X ~ Bernoulli(p). This can be generalized to generate Y ~ Binomial(n, p) by generating U 1, U 2, … U n. Setting X i as above and let Y = X 1 + ∙∙∙ + X n.

28. STA347 - week 928 Simulating Discrete Distributions In general, suppose we wish to generate a random variable with probability mass function p. Let, x 1 0. Let U ~ Uniform[0, 1]. Define Y by: Theorem 1: Y is a discrete random variable, having probability mass function p. Proof:

29. STA347 - week 929 Simulating Continuous Distributions - Example Suppose we wish to generate X ~ Uniform[a, b]. We start by generating U ~ Uniform[0, 1] and then set: Using one-dimensional change of variable theorem we can easily show that X ~ Uniform[a, b].

30. STA347 - week 930 Simulating Continuous Distributions In general, simulating continuous distribution is not an easy task. However, for certain continuous distributions it is not difficult. The general method for simulating continuous distribution makes use of the inverse cumulative distribution function. The inverse cdf of a random variable X with cumulative distribution function F is defined by: for 0 < t < 1.

31. STA347 - week 931 Inversion Method for Generating RV Let F be any cumulative distribution function, and let U ~ Uniform[0, 1]. Define a random variable Y by: Theorem 2: Y has cumulative distribution function given by F. That is, Proof:

32. STA347 - week 932 Important Notes The theorem above is valid for any cumulative distribution function whether it corresponds to a continuous distribution, a discrete distribution or a mixture of the two. The inversion method for generating random variables described above can be used whenever the distribution function is not too complicated and has a close form. For distributions that are too complicated to sample using the inversion method and for which there is no simple trick, it may still be possible to generate samples using Markov chain methods.

33. STA347 - week 933 Example – Exponential Distribution Suppose X ~ Exponential(λ). The probability density function of X is: The cdf of X is: Setting and solving for x we get… Therefore, by theorem 2 above, where U ~ Uniform[0, 1], has an Exponential(λ) distribution. 33

34. STA347 - week 934 Example – Standard Normal Distribution Suppose X ~ Normal(0,1). The cdf of X is denoted by Ф(x). It is given by: Then, if U ~ Uniform[0, 1], by theorem 2 above has a N(0,1) distribution. However, since both Ф and Ф -1 don’t have a close form, i.e., it is difficult to compute them, the inversion method for generating RV is not practical. 34