1. Probability theory 2010 Main topics in the course on probability theory  Multivariate random variables  Conditional distributions  Transforms  Order variables  The multivariate normal distribution  The exponential family of distributions  Convergence in probability and distribution

2. Probability theory 2010 Objectives  Provide a solid understanding of major concepts in probability theory  Increase the ability to derive probabilistic relationships in given probability models  Facilitate reading scientific articles on inference based on probability models

3. Probability theory 2010 Joint distribution function provides a complete description of the two-dimensional distribution of the random vector ( X, Y )

4. Probability theory 2010 Joint distribution function

5. Probability theory 2010 Joint probability density

6. Probability theory 2010 Joint probability function

7. Probability theory 2010 Marginal distributions Marginal probability density of X

8. Probability theory 2010 Independence Independent events Independent stochastic variables Sufficient that

9. Probability theory 2010 Covariance Assume that E(X) = E(Y) = 0. Then, E(XY) can be regarded as a measure of covariance between X and Y More generally, we set Cov(X, Y) = 0 if X and Y are independent. The converse need not be true.

10. Probability theory 2010 Covariance rules

11. Probability theory 2010 Covariance and correlation Scale-invariant covariance

12. Probability theory 2010 Inequalities Proof: Assume that Then, observe that

13. Probability theory 2010 Functions of random variables Let Y = a + bX Derive the relationship between the probability density functions of Y and X

14. Probability theory 2010 Functions of random variables Let X be uniformly distributed on (0,1) and set Derive the probability density function of Y

15. Probability theory 2010 Functions of random variables Let X have an arbitrary continuous distribution, and suppose that g is a (differentiable) strictly increasing function. Set Then and

16. Probability theory 2010 Linear functions of random vectors Let (X 1, X 2 ) have a uniform distribution on D = {(x, y); 0 < x <1, 0 < y <1} Set Then.

17. Probability theory 2010 Functions of random vectors Let (X 1, X 2 ) have an arbitrary continuous distribution, and suppose that g is a (differentiable) one-to-one transformation. Set Then where h is the inverse of g. Proof: Use the variable transformation theorem

18. Probability theory 2010 Random number generation  Uniform distribution  Bin(2; 0.5)  Po(4)  Exp(1)

19. Probability theory 2010 Random number generation - the inversion method  Let F denote the cumulative distribution function of a probability distribution.  Let Z be uniformly distributed on the interval (0,1)  Then, X = F -1 (Z) will have the cumulative distribution function F.  How can we generate normally distributed random numbers?

20. Probability theory 2010 Random number generation: method 3 ( the envelope-rejection method)  Generate x from a probability density g(x) such that cg(x)  f(x)  Draw u from a uniform distribution on (0,1)  Accept x if u < f(x)/cg(x) *************************** Justification: Let X denote a random number from the probability density g. Then  How can we generate normally distributed random numbers?

21. Probability theory 2010 Random number generation - LCGs  Linear congruential generators are defined by the recurrence relation  Numerical Recipes in C advocates a generator of this form with: a = 1664525, b = 1013904223, M = 2 32 Drawback: Serial correlation

22. Probability theory 2010 Exercises: Chapter I 1.3, 1.8, 1.14, 1.18, 1.30, 1.31, 1.33