1. Probability theory 2011 Main topics in the course on probability theory  The concept of probability – Repetition of basic skills  Multivariate random variables – Chapter 1  Conditional distributions – Chapter 2  Transforms – Chapter 3  Order variables – Chapter 4  The multivariate normal distribution – Chapter 5  The exponential family of distributions - Slides  Convergence in probability and distribution – Chapter 6

2. Probability theory 2011 Objectives  Provide a solid understanding of major concepts in probability theory  Increase the ability to derive probabilistic relationships in given probability models  Facilitate reading scientific articles on inference based on probability models

3. Probability theory 2011 The concept of probability – Repetition of basic skills  “Gut: Introduction” + More  Whiteboard

4. Probability theory 2011 Multivariate random variables  Gut: Chapter 1  Slides

5. Probability theory 2011 Joint distribution function - Copula provides a complete description of the two-dimensional distribution of the random vector ( X, Y )

6. Probability theory 2011 Joint distribution function

7. Probability theory 2011 Joint probability density

8. Probability theory 2011 Joint probability function

9. Probability theory 2011 Marginal distributions Marginal probability density of X

10. Probability theory 2011 Independence Independent events Independent stochastic variables Sufficient that

11. Probability theory 2011 Covariance Assume that E(X) = E(Y) = 0. Then, E(XY) can be regarded as a measure of covariance between X and Y More generally, we set Cov(X, Y) = 0 if X and Y are independent. The converse need not be true.

12. Probability theory 2011 Covariance rules

13. Probability theory 2011 Covariance and correlation Scale-invariant covariance

14. Probability theory 2011 Inequalities Proof: Assume that Then, observe that

15. Probability theory 2011 Functions of random variables Let Y = a + bX Derive the relationship between the probability density functions of Y and X

16. Probability theory 2011 Functions of random variables Let X be uniformly distributed on (0,1) and set Derive the probability density function of Y

17. Probability theory 2011 Functions of random variables Let X have an arbitrary continuous distribution, and suppose that g is a (differentiable) strictly increasing function. Set Then and

18. Probability theory 2011 Linear functions of random vectors Let (X 1, X 2 ) have a uniform distribution on D = {(x, y); 0 < x <1, 0 < y <1} Set Then

19. Probability theory 2011 Functions of random vectors Let (X 1, X 2 ) have an arbitrary continuous distribution, and suppose that g is a (differentiable) one-to-one transformation. Set Then where h is the inverse of g. Proof: Use the variable transformation theorem

20. Probability theory 2011 Random number generation  Uniform distribution  Bin(2; 0.5)  Po(4)  Exp(1)

21. Probability theory 2011 Random number generation - the inversion method  Let F denote the cumulative distribution function of a probability distribution.  Let Z be uniformly distributed on the interval (0,1)  Then, X = F -1 (Z) will have the cumulative distribution function F.  How can we generate normally distributed random numbers?

22. Probability theory 2011 Random number generation: method 3 ( the envelope-rejection method)  Generate x from a probability density g(x) such that cg(x)  f(x)  Draw u from a uniform distribution on (0,1)  Accept x if u < f(x)/cg(x) *************************** Justification: Let X denote a random number from the probability density g. Then  How can we generate normally distributed random numbers?

23. Probability theory 2011 Random number generation - LCGs  Linear congruential generators are defined by the recurrence relation  Numerical Recipes in C advocates a generator of this form with: a = 1664525, b = 1013904223, M = 2 32 Drawback: Serial correlation

24. Probability theory 2011 Exercises: Chapter I 1.3, 1.8, 1.14, 1.18, 1.30, 1.31, 1.33