1. 1 11 Lecture 12 Overview of Probability and Random Variables (I) Fall 2008 NCTU EE Tzu-Hsien Sang

2. 2 Outline What is Probability Random Variables Distribution Functions Multiple Variables Statistical Averages

3. 3 What is Probability Why probabilistic analysis? My answer: to To hold on something in a world full of chaos and uncertainties Applications Engineering Systems and control (e.g., aircraft control), Decision and resource allocation under uncertainty (e.g., communications networks), Reliability (noise, error control, failures) Economics and finance Physics, statistical mechanics, thermodynamics Medicine, FDA, drugs and procedures Linguistics, automatic speech recognition and translation

4. 4 In this course, both messages (signals) & noises can be treated as random in nature. Q: What benefits can we get from doing that? Some definitions: (a)random variable (r.v.): one random quantity (b)random sequence: sequence of random variable (c)random process: a (continuous-time) function whose value (at any time instant) is a r.v.

5. 5 A Basic Probabilistic model: an experiment means a mathematical model of a process with an outcome that is not fully predictable Basic Components: Outcome: each experiment produces exactly one outcome Sample space: A list of all outcomes of an experiment Algebra of events (set theory): language for manipulating collections of elementary events = sets Probability law: means of assigning a probability to events in a consistent and useful way

6. 6

7. 7 How to establish a probabilistic model? Relative Frequency --- experimental, intuitive, Axiomatic Theory --- mathematical, rigorous, facilitate further derivation (most importantly, it can deal with infinity!) Example: Tossing two fair coins (Please list the events, sample space, probability law, etc.)

8. 8 Probability Theory (I)

9. 9 Probability Theory (II)

10. 10 Math Models and Reality Models are purely mathematical creations. There is no guarantee that they can “fit” the reality well enough. You need to balance among accuracy, simplicity, and tractability. For many cases, difficulties arise from fuzzy word usage in the formulation of the problem. Example Bertrand’s “paradox” (1889) Consider a “randomly drawn” chord of a circle. What is the probability that its length is greater than a certain length?

11. 11 Random Variables

12. 12

13. 13 Distribution Functions

14. 14

15. 15 Example pdfs Binomial distribution

16. 16

17. 17 Gaussian (normal) distribution

18. 18 Multiple Variables

19. 19

20. 20 Conditional Probability: a derived probability measure Conditional cdf and pdf: Conditional random variables:

21. 21

22. 22 Bayes’ Theorem Objective: Inference instead of prediction or observation Given observed “effect" or “result", infer the unobserved “cause". Assume we know the “prior" or “a priori" probabilities and the conditional probabilities in order to compute the “posterior" or “a posteriori" probabilities.