1. Statistics review of basic probability and statistics

2. Statistics Outline –Introduction –Random Variables –Simulation Output 040669 || WS 2008 || Dr. Verena Schmid || PR KFK PM/SCM/TL Praktikum Simulation I 2

3. Statistics integral part of simulation studys –model probabilistic systems –validate simulation model –choose input probability distributions –generate random samples –perform statistical analysis 040669 || WS 2008 || Dr. Verena Schmid || PR KFK PM/SCM/TL Praktikum Simulation I 3

4. Random Variables density and distribution functions 040669 || WS 2008 || Dr. Verena Schmid || PR KFK PM/SCM/TL Praktikum Simulation I4

5. Experiments experiment –process whose output is not known with certainty –sample space (S): set of all possible outcomes –sample points: outcomes in the sample space examples: –flipping a coin: S = {H, T} –toss a die: S = {1,2,3,4,5,6} random variable –assigns a real number to each point in the sample space 040669 || WS 2008 || Dr. Verena Schmid || PR KFK PM/SCM/TL Praktikum Simulation I 5

6. Random Variable discrete random variables –a random variable X is said to be discrete if it can take on at most a countable number of values x 1, x 2, x 3,..,x n –probability mass function: the probability that X takes on the value x i is given by –probability that X lies in Interval I [a,b] –(cumulative) distribution function 040669 || WS 2008 || Dr. Verena Schmid || PR KFK PM/SCM/TL Praktikum Simulation I 6

7. x F(x) 1.0 0.9 0.8 0.7 0.6 0.5 0.4 0.3 0.2 0.1 0.0 0 1 2 3 45 discrete random variables (example) X takes on values 1,2,3,4 with probabilities 1/6, 1/3, 1/3, 1/6 probability mass functiondistribution function 040669 || WS 2008 || Dr. Verena Schmid || PR KFK PM/SCM/TL Praktikum Simulation I 7 p(x)p(x) 12 3 4 0.35 0.30 0.25 0.20 0.15 0.10 0.05 0.00 x

8. Random Variable (cont.) continuous random variables –a random variable X is said to be continuous if it can take on an uncountable infinite number of different values –probability density function: nonnegative function for any set of real numbers B (ex. B = [1,2]) its not the probability a continuous random variable X equals x different interpretation (for ¢ x > 0) 040669 || WS 2008 || Dr. Verena Schmid || PR KFK PM/SCM/TL Praktikum Simulation I 8

9. continuous random variable (cont.) distribution function –probability random variable X is less or equal to x –probability that X lies in interval I = [a,b] 040669 || WS 2008 || Dr. Verena Schmid || PR KFK PM/SCM/TL Praktikum Simulation I 9

10. continuous random variable (cont.) 040669 || WS 2008 || Dr. Verena Schmid || PR KFK PM/SCM/TL Praktikum Simulation I 10 x f (x) interpretation of probability density & distribution function

11. special continuous distribution: uniform distribution 040669 || WS 2008 || Dr. Verena Schmid || PR KFK PM/SCM/TL Praktikum Simulation I 11 probability density function f(x) distribution function F(x) f(x) x 0 1 1 F(x) x 0 1 1

12. measures expected value (mean) –measure of central tendency –center of gravity variance –measure of the dispersion around mean 040669 || WS 2008 || Dr. Verena Schmid || PR KFK PM/SCM/TL Praktikum Simulation I 12

13. small vs. large variance density functions 040669 || WS 2008 || Dr. Verena Schmid || PR KFK PM/SCM/TL Praktikum Simulation I 13 µ µ σ 2 large σ 2 small X X X X

14. more measures covariance –measure of linear dependence of random variables X i and Y j 040669 || WS 2008 || Dr. Verena Schmid || PR KFK PM/SCM/TL Praktikum Simulation I 14

15. more measures (cont) correlation –measure of linear dependence of random variables X i and Y j –dimensionless (as opposed to covariance) 040669 || WS 2008 || Dr. Verena Schmid || PR KFK PM/SCM/TL Praktikum Simulation I 15

16. example distributions 040669 || WS 2008 || Dr. Verena Schmid || PR KFK PM/SCM/TL Praktikum Simulation I16

17. discrete uniform distribution A random variable that has any of n possible values k 1,k 2,...,k m that are equally probable, has a discrete uniform distribution Parameter m example: –number of points you’ll get on the top layer if you toss one dice –n (number of possible values) = 6 –k i = i (i = 1,…, 6) 040669 || WS 2008 || Dr. Verena Schmid || PR KFK PM/SCM/TL Praktikum Simulation I 17

18. binomial distribution A typical example is the following: assume 5% of the population is HIV- positive. You pick 500 people randomly. How likely is it that you get 30 or more HIV-positives? –Parameters n and p 040669 || WS 2008 || Dr. Verena Schmid || PR KFK PM/SCM/TL Praktikum Simulation I 18

19. normal distribution N( ,  ) 040669 || WS 2008 || Dr. Verena Schmid || PR KFK PM/SCM/TL Praktikum Simulation I 19 probability density function f(x) also called Gaussian distribution - extremely important probability distribution in many fields distribution function F(x)

20. poisson distribution Distribution of “rare” events –Parameters usually used for modeling number of arrivals per time unit 040669 || WS 2008 || Dr. Verena Schmid || PR KFK PM/SCM/TL Praktikum Simulation I 20

21. geometric distribution For example, suppose an ordinary die is thrown repeatedly until the first time a "1" appears. The probability distribution of the number of times it is thrown is supported on the infinite set { 1, 2, 3,... } and is a geometric distribution with p = 1/6. –Parameter p 040669 || WS 2008 || Dr. Verena Schmid || PR KFK PM/SCM/TL Praktikum Simulation I 21

22. negative binomial distribution For example, suppose an ordinary die is thrown repeatedly until the “r”-th time a "1" appears. The probability distribution of the number of times it is thrown is supported on the infinite set { 1, 2, 3,... }. –Parameters k and p 040669 || WS 2008 || Dr. Verena Schmid || PR KFK PM/SCM/TL Praktikum Simulation I 22

23. hypergeometric distribution A typical example is the following: There is a shipment of N objects in which M are defective. The hyper- geometric distribution describes the probability that in a sample of n distinctive objects drawn from the shipment exactly k objects are defective. –Parameters N, M, and n 040669 || WS 2008 || Dr. Verena Schmid || PR KFK PM/SCM/TL Praktikum Simulation I 23

24. 24 normal distribution N( ,  ) N(0;1) – standard normal distribution Standardizing normal random variable X Under certain conditions, the distribution of a sum of a large number of independent variables is approximately normal (central limit theorem). The practical importance of the central limit theorem is that the normal distribution can be used as an approximation to some other distributions. –A binomial distribution with parameters n and p is approximately normal for large n and p not too close to 1 or 0. The approximating normal distribution has mean μ = np and variance σ2 = np(1 − p). –A Poisson distribution with parameter λ is approximately normal for large λ. The approximating normal distribution has mean μ = λ and variance σ 2 = λ.

25. uniform distribution 040669 || WS 2008 || Dr. Verena Schmid || PR KFK PM/SCM/TL Praktikum Simulation I 25 probability density function f(x) each result in interval [0,1] has the same probability distribution function F(x) 1 01 01

26. uniform distribution 040669 || WS 2008 || Dr. Verena Schmid || PR KFK PM/SCM/TL Praktikum Simulation I 26 probability density function f(x) each result in interval [a,b] has the same probability distribution function F(x)

27. exponential distribution 040669 || WS 2008 || Dr. Verena Schmid || PR KFK PM/SCM/TL Praktikum Simulation I 27 probability density function f(x) often used for modeling time between 2 events distribution function F(x)

28. erlang distribution 040669 || WS 2008 || Dr. Verena Schmid || PR KFK PM/SCM/TL Praktikum Simulation I 28 probability density function f(x) modeling sum of n exponential random variables distribution function F(x)

29. Estimation 040669 || WS 2008 || Dr. Verena Schmid || PR KFK PM/SCM/TL Praktikum Simulation I29

30. Estimation of Mean X 1, X 2, … X n are IID random variables –finite population –true mean ¹ –true variance ¾ ² sample mean (estimator for ¹ ) sample variance (estimator for ¾ ²) 040669 || WS 2008 || Dr. Verena Schmid || PR KFK PM/SCM/TL Praktikum Simulation I 30 identically and independently distributed

31. Problem: no way (without additional information) to see how close estimator X(n) is to real mean → i.e. how “good” is estimator Estimation 040669 || WS 2008 || Dr. Verena Schmid || PR KFK PM/SCM/TL Praktikum Simulation I 31 xx first observation of X(n)second observation of X(n)

32. Estimation X(n) itself is a random variable –random output of a single simulation run –on one experiment may be close to real ¹ –on another it my differ by a large amount from ¹ –has its own variance!!! –the large sample size n → the smaller variance of estimator → the closer estimator will be to true mean 040669 || WS 2008 || Dr. Verena Schmid || PR KFK PM/SCM/TL Praktikum Simulation I 32

33. Estimation: construct confidence interval central limit theorem says… for large sample size n the estimator (sample mean) X(n) is approximately normally distributed with mean ¹ and variance ¾ ²/n problem real variance ¾ ² is usually unknown (but sample variance S²(n) converges to ¾ ² as n is “large enough”) 040669 || WS 2008 || Dr. Verena Schmid || PR KFK PM/SCM/TL Praktikum Simulation I 33

34. confidence intervals confidence interval for the mean (unknown variance) t n-1, 1- ® /2 is the upper 1- ® /2 critical point of the t distribution with n-1 degrees of freedom → (tables!) confidence interval for the mean (unknown mean, normal distribution) 040669 || WS 2008 || Dr. Verena Schmid || PR KFK PM/SCM/TL Praktikum Simulation I 34

35. confidence intervals interpretation: nothing probabilistic about confidence intervals (once they have been created – just numbers) if one constructs a very large number of independent confidence intervals, each based on n (“sufficiently large”) observations, the proportion of these confidence intervals that cover the real ¹ is equal to 1 - ® 040669 || WS 2008 || Dr. Verena Schmid || PR KFK PM/SCM/TL Praktikum Simulation I 35

36. example: confidence intervals 10 observations 1.2 1.5 1.68 1.89 0.95 1.49 1.58 1.55 0.5 1.09 sample size n = 10 estimator for mean X(10) = 1.343 estimator for variance S²(10) = 0.16751222 90% confidence interval ( ® = 10%) 040669 || WS 2008 || Dr. Verena Schmid || PR KFK PM/SCM/TL Praktikum Simulation I 36

37. necessary sample size Based on a sample with size n 0 –calculate the confidence interval for the mean –identify half-width h 0 How many sample values n would be necessary –to achieve a half-width h ??? 040669 || WS 2008 || Dr. Verena Schmid || PR KFK PM/SCM/TL Praktikum Simulation I 37

38. example (cont) 10 observations 1.2 1.5 1.68 1.89 0.95 1.49 1.58 1.55 0.5 1.09 based on n 0 = 10 observations –half width h 0 = 0.24 Q: if you’d prefer a half width of 0.12 –what’s the necessary number of observation?? 040669 || WS 2008 || Dr. Verena Schmid || PR KFK PM/SCM/TL Praktikum Simulation I 38

39. hypothesis testing test for real ¹ (two-tailed hypothesis testing) H 0 ¹ = ¹ 0 ( ¹ 0 fixed hypothesized value for ¹ ) alternatively if ¹ 0 is not contained in (1- ® )% confidence interval → reject H 0 if ¹ 0 is contained in (1 - ® )% confidence interval → accept H 0 040669 || WS 2008 || Dr. Verena Schmid || PR KFK PM/SCM/TL Praktikum Simulation I 39

40. Comparison of 2 Alternatives Confidence Intervals for the difference of the means (independent) –equal variances, but unknown –different variances 040669 || WS 2008 || Dr. Verena Schmid || PR KFK PM/SCM/TL Praktikum Simulation I 40