Tests of Random Number Generators

Slides:



Advertisements
Similar presentations
1 Chi-Square Test -- X 2 Test of Goodness of Fit.
Advertisements

Hypothesis testing Another judgment method of sampling data.
Lecture (11,12) Parameter Estimation of PDF and Fitting a Distribution Function.
Random Number Generation. Random Number Generators Without random numbers, we cannot do Stochastic Simulation Most computer languages have a subroutine,
CHAPTER 21 Inferential Statistical Analysis. Understanding probability The idea of probability is central to inferential statistics. It means the chance.
Outline input analysis input analyzer of ARENA parameter estimation
6-1 Introduction To Empirical Models 6-1 Introduction To Empirical Models.
Statistical Significance What is Statistical Significance? What is Statistical Significance? How Do We Know Whether a Result is Statistically Significant?
1 Test a hypothesis about a mean Formulate hypothesis about mean, e.g., mean starting income for graduates from WSU is $25,000. Get random sample, say.
Statistical Significance What is Statistical Significance? How Do We Know Whether a Result is Statistically Significant? How Do We Know Whether a Result.
Hypothesis : Statement about a parameter Hypothesis testing : decision making procedure about the hypothesis Null hypothesis : the main hypothesis H 0.
Random Number Generation
Aaker, Kumar, Day Seventh Edition Instructor’s Presentation Slides
1 Simulation Modeling and Analysis Pseudo-Random Numbers.
Properties of Random Numbers
5-3 Inference on the Means of Two Populations, Variances Unknown
Nonparametrics and goodness of fit Petter Mostad
Inferential Statistics
Fall 2011 CSC 446/546 Part 6: Random Number Generation.
Choosing Statistical Procedures
+ Quantitative Statistics: Chi-Square ScWk 242 – Session 7 Slides.
ETM 607 – Random Number and Random Variates
AM Recitation 2/10/11.
Aaker, Kumar, Day Ninth Edition Instructor’s Presentation Slides
Overview of Statistical Hypothesis Testing: The z-Test
The paired sample experiment The paired t test. Frequently one is interested in comparing the effects of two treatments (drugs, etc…) on a response variable.
Go to Index Analysis of Means Farrokh Alemi, Ph.D. Kashif Haqqi M.D.
The Examination of Residuals. The residuals are defined as the n differences : where is an observation and is the corresponding fitted value obtained.
Copyright © 2012 Wolters Kluwer Health | Lippincott Williams & Wilkins Chapter 17 Inferential Statistics.
Chapter 15 Data Analysis: Testing for Significant Differences.
Chi-square Test of Independence Hypotheses Neha Jain Lecturer School of Biotechnology Devi Ahilya University, Indore.
Learning Objectives In this chapter you will learn about the t-test and its distribution t-test for related samples t-test for independent samples hypothesis.
Chapter 7 Random-Number Generation
Tests for Random Numbers Dr. Akram Ibrahim Aly Lecture (9)
CS433: Modeling and Simulation Dr. Anis Koubâa Al-Imam Mohammad bin Saud University 15 October 2010 Lecture 05: Statistical Analysis Tools.
Maximum Likelihood Estimator of Proportion Let {s 1,s 2,…,s n } be a set of independent outcomes from a Bernoulli experiment with unknown probability.
Basic Concepts in Number Theory Background for Random Number Generation 1.For any pair of integers n and m, m  0, there exists a unique pair of integers.
The Examination of Residuals. Examination of Residuals The fitting of models to data is done using an iterative approach. The first step is to fit a simple.
1/23 Ch10 Nonparametric Tests. 2/23 Outline Introduction The sign test Rank-sum tests Tests of randomness The Kolmogorov-Smirnov and Anderson- Darling.
Inference and Inferential Statistics Methods of Educational Research EDU 660.
Confidence intervals and hypothesis testing Petter Mostad
Chapter 8 Introduction to Hypothesis Testing ©. Chapter 8 - Chapter Outcomes After studying the material in this chapter, you should be able to: 4 Formulate.
CHI SQUARE TESTS.
Selecting Input Probability Distribution. Simulation Machine Simulation can be considered as an Engine with input and output as follows: Simulation Engine.
Section Copyright © 2014, 2012, 2010 Pearson Education, Inc. Lecture Slides Elementary Statistics Twelfth Edition and the Triola Statistics Series.
GG 313 Lecture 9 Nonparametric Tests 9/22/05. If we cannot assume that our data are at least approximately normally distributed - because there are a.
Testing the Differences between Means Statistics for Political Science Levin and Fox Chapter Seven 1.
Virtual University of Pakistan Lecture No. 44 of the course on Statistics and Probability by Miss Saleha Naghmi Habibullah.
Analyzing Statistical Inferences July 30, Inferential Statistics? When? When you infer from a sample to a population Generalize sample results to.
© Copyright McGraw-Hill 2004
Chapter 22 Comparing Two Proportions.  Comparisons between two percentages are much more common than questions about isolated percentages.  We often.
1 1 Slide © 2008 Thomson South-Western. All Rights Reserved Chapter 12 Tests of Goodness of Fit and Independence n Goodness of Fit Test: A Multinomial.
Environmental Modeling Basic Testing Methods - Statistics II.
Statistical Inference for the Mean Objectives: (Chapter 8&9, DeCoursey) -To understand the terms variance and standard error of a sample mean, Null Hypothesis,
Lesson Runs Test for Randomness. Objectives Perform a runs test for randomness Runs tests are used to test whether it is reasonable to conclude.
MONTE CARLO METHOD DISCRETE SIMULATION RANDOM NUMBER GENERATION Chapter 3 : Random Number Generation.
Non-parametric Tests Research II MSW PT Class 8. Key Terms Power of a test refers to the probability of rejecting a false null hypothesis (or detect a.
CHI SQUARE DISTRIBUTION. The Chi-Square (  2 ) Distribution The chi-square distribution is the probability distribution of the sum of several independent,
Section Copyright © 2014, 2012, 2010 Pearson Education, Inc. Lecture Slides Elementary Statistics Twelfth Edition and the Triola Statistics Series.
Hypothesis Testing I The One-sample Case
STAT 312 Chapter 7 - Statistical Intervals Based on a Single Sample
Lecture Slides Elementary Statistics Twelfth Edition
Hypothesis Theory examples.
Chapter 7 Random Number Generation
6-1 Introduction To Empirical Models
Properties of Random Numbers
Discrete Event Simulation - 4
The Examination of Residuals
Lecture Slides Elementary Statistics Twelfth Edition
Random Number Generation
Presentation transcript:

Tests of Random Number Generators Empirical tests (frequency test) Uniformity Compute sample moments Goodness of fit Independence Gap Test Runs Test Poker Test Spectral Test Autocorrelation Test

Empirical tests Frequency test. Uses the Kolmogorov-Smirnov or the chi-square test to compare the distribution of the set of numbers generated to a uniform distribution. Kolmogorov-Smirnov Goodness-of-fit Test 30

Test for Random Numbers (cont.) Uniformity test In testing for uniformity, the hypotheses are as follows: H0: Ri ~ U[0,1] H1: Ri ¹ U[0,1] The null hypothesis, H0, reads that the numbers are distributed uniformly on the interval [0,1]. Random Numbers 30

Kolmogorov-Smirnov Goodness-of-fit Test:

Test for Random Numbers (cont.) Level of significance a a = P(reject H0 | H0 true) Frequently, a is set to 0.01 or 0.05 (Hypothesis) Actually True Actually False Accept 1 - a b (Type II error) Reject a 1 - b (Type I error) Random Numbers 30

Example

Testing the Random Numbers (independent test) 1. Runs test. Tests the runs up and down or the runs above and below the mean by comparing the actual values to expected values. The statistic for comparison is the chi-square. 2. Autocorrelation test. Tests the correlation between numbers and compares the sample correlation to the expected correlation of zero. 3. Gap test. Counts the number of digits that appear between repetitions of a particular digit and then uses the Kolmogorov-Smirnov test to compare with the expected number of gaps. 4. Poker test. Treats numbers grouped together as a poker hand. Then the hands obtained are compared to what is expected using the chi-square test. 30

Independent test In testing for independence, the hypotheses are as follows; H0: Ri ~ independently H1: Ri ¹ independently This null hypothesis, H0, reads that the numbers are independent. Failure to reject the null hypothesis means that no evidence of dependence has been detected on the basis of this test. This does not imply that further testing of the generator for independence is unnecessary. Random Numbers 30

* Test Statistic: S = runs of numbers above or below median) Independence — Sign Test * Test Statistic: S = runs of numbers above or below median) * For large N, S is distributed N( = 1+(N/2), 2 = N/2) Example N = 15, S = 7, distributed N( = 8.5, 2 = 15/2) Maximum value for S: N (negative dependency) Minimum value for S: 1 (positive dependency) .87 .15 .23 .45 .69 .32 .30 .19 .24 .18 .65 .82 .93 .22 .81 + - - - + - - - - - + + + - +

Reject H0 in favor of HA if Normal Curve Rejection Regions REJECT (+ve) REJECT (-ve) Do Not REJECT H0 : Independence HA : Dependence -Z/2 Z/2 Reject H0 in favor of HA if Z = (S - (1+(N/2))) / (N/2)1/2  Z/2 or Z Z/2

Test for Random Numbers (cont.) The Gap Test measures the number of digits between successive occurrences of the same digit. (Example) length of gaps associated with the digit 3. 4, 1, 3, 5, 1, 7, 2, 8, 2, 0, 7, 9, 1, 3, 5, 2, 7, 9, 4, 1, 6, 3 3, 9, 6, 3, 4, 8, 2, 3, 1, 9, 4, 4, 6, 8, 4, 1, 3, 8, 9, 5, 5, 7 3, 9, 5, 9, 8, 5, 3, 2, 2, 3, 7, 4, 7, 0, 3, 6, 3, 5, 9, 9, 5, 5 5, 0, 4, 6, 8, 0, 4, 7, 0, 3, 3, 0, 9, 5, 7, 9, 5, 1, 6, 6, 3, 8 8, 8, 9, 2, 9, 1, 8, 5, 4, 4, 5, 0, 2, 3, 9, 7, 1, 2, 0, 3, 6, 3 Note: eighteen 3’s in list ==> 17 gaps, the first gap is of length 10 Random Numbers 30

Test for Random Numbers (cont.) We are interested in the frequency of gaps. P(gap of 10) = P(not 3) ××× P(not 3) P(3) , note: there are 10 terms of the type P(not 3) = (0.9)10 (0.1) The theoretical frequency distribution for randomly ordered digit is given by F(x) = 0.1 (0.9)n = 1 - 0.9x+1 Note: observed frequencies for all digits are compared to the theoretical frequency using the Kolmogorov-Smirnov test. Random Numbers 30

Test for Random Numbers (cont.) (Example) Based on the frequency with which gaps occur, analyze the 110 digits above to test whether they are independent. Use a= 0.05. The number of gaps is given by the number of digits minus 10, or 100. The number of gaps associated with the various digits are as follows: Digit 0 1 2 3 4 5 6 7 8 9 # of Gaps 7 8 8 17 10 13 7 8 9 13 Random Numbers 30

Test for Random Numbers (cont.) Gap Test Example Relative Cum. Relative Gap Length Frequency Frequency Frequency F(x) |F(x) - SN(x)| 0-3 35 0.35 0.35 0.3439 0.0061 4-7 22 0.22 0.57 0.5695 0.0005 8-11 17 0.17 0.74 0.7176 0.0224 12-15 9 0.09 0.83 0.8147 0.0153 16-19 5 0.05 0.88 0.8784 0.0016 20-23 6 0.06 0.94 0.9202 0.0198 24-27 3 0.03 0.97 0.9497 0.0223 28-31 0 0.00 0.97 0.9657 0.0043 32-35 0 0.00 0.97 0.9775 0.0075 36-39 2 0.02 0.99 0.9852 0.0043 40-43 0 0.00 0.99 0.9903 0.0003 44-47 1 0.01 1.00 0.9936 0.0064 Random Numbers 30

Test for Random Numbers (cont.) The critical value of D is given by D0.05 = 1.36 / Ö100 = 0.136 Since D = max |F(x) - SN(x)| = 0.0224 is less than D0.05, do not reject the hypothesis of independence on the basis of this test. Random Numbers 30

Test for Random Numbers (cont.) Run Tests (Up and Down) Consider the 40 numbers; both the Kolmogorov-Smirnov and Chi-square would indicate that the numbers are uniformly distributed. But, not so. 0.08 0.09 0.23 0.29 0.42 0.55 0.58 0.72 0.89 0.91 0.11 0.16 0.18 0.31 0.41 0.53 0.71 0.73 0.74 0.84 0.02 0.09 0.30 0.32 0.45 0.47 0.69 0.74 0.91 0.95 0.12 0.13 0.29 0.36 0.38 0.54 0.68 0.86 0.88 0.91 Random Numbers 30

Test for Random Numbers (cont.) Now, rearrange and there is less reason to doubt independence. 0.41 0.68 0.89 0.84 0.74 0.91 0.55 0.71 0.36 0.30 0.09 0.72 0.86 0.08 0.54 0.02 0.11 0.29 0.16 0.18 0.88 0.91 0.95 0.69 0.09 0.38 0.23 0.32 0.91 0.53 0.31 0.42 0.73 0.12 0.74 0.45 0.13 0.47 0.58 0.29 Random Numbers 30

Test for Random Numbers (cont.) Concerns: Number of runs Length of runs Note: If N is the number of numbers in a sequence, the maximum number of runs is N-1, and the minimum number of runs is one. If “a” is the total number of runs in a sequence, the mean and variance of “a” is given by Random Numbers 30

Test for Random Numbers (cont.) ma = (2n - 1) / 3 = (16N - 29) / 90 For N > 20, the distribution of “a” approximated by a normal distribution, N(ma , ). This approximation can be used to test the independence of numbers from a generator. Z0 = (a - ma) / sa Random Numbers 30

Test for Random Numbers (cont.) Substituting for ma and sa ==> Za = {a - [(2N-1)/3]} / {Ö(16N-29)/90}, where Z ~ N(0,1) Acceptance region for hypothesis of independence -Za/2 £ Z0 £ Za/2 Random Numbers 30

Test for Random Numbers (cont.) (Example) Based on runs up and runs down, determine whether the following sequence of 40 numbers is such that the hypothesis of independence can be rejected where a = 0.05. 0.41 0.68 0.89 0.94 0.74 0.91 0.55 0.62 0.36 0.27 0.19 0.72 0.75 0.08 0.54 0.02 0.01 0.36 0.16 0.28 0.18 0.01 0.95 0.69 0.18 0.47 0.23 0.32 0.82 0.53 0.31 0.42 0.73 0.04 0.83 0.45 0.13 0.57 0.63 0.29 Random Numbers 30

Test for Random Numbers (cont.) The sequence of runs up and down is as follows: + + + - + - + - - - + + - + - - + - + - - + - - + - + + - - + + - + - - + + - There are 26 runs in this sequence. With N=40 and a=26, ma = {2(40) - 1} / 3 = 26.33 and = {16(40) - 29} / 90 = 6.79 Then, Z0 = (26 - 26.33) / Ö(6.79) = -0.13 Now, the critical value is Z0.025 = 1.96, so the independence of the numbers cannot be rejected on the basis of this test. Random Numbers 30

Test for Random Numbers (cont.) Poker Test - based on the frequency with which certain digits are repeated. Example: 0.255 0.577 0.331 0.414 0.828 0.909 Note: a pair of like digits appear in each number generated. Random Numbers 30