1. Chapter 7 Random Number Generation
Banks, Carson, Nelson & Nicol
Discrete-Event System Simulation

2. Properties of Random Numbers
Two important statistical properties:
Uniformity
Independence.
Random Number, Ri, must be independently drawn from a uniform distribution with pdf, given by f(x)
The expected value of each Ri is given by E(R)
The variance is given by V(R) = x2dx – [E(R)]2 = 1/12

3. Generation of Random Numbers
Important considerations in RN routines:
Fast
Portable to different computers and diff programming languages
Have sufficiently long cycle, before the numbers start repeating.
Replicable. If a starting point of RNs or a condition is given, we should be able to generate the same set of RNs (for debugging)
Closely approximate the ideal statistical properties of uniformity and independence.

4. Generation of pseudo-random numbers
“Pseudo” means false. Generating random numbers using a known method removes the potential for true randomness.
If the method is known the random numbers can be replicated.
Goal: To produce a sequence of numbers in [0,1] that simulates, or imitates, the ideal properties of random numbers (RN).

5. Characteristics of a Good Generator
Maximum Density - Such that he values assumed by Ri, i = 1,2,…, leave no large gaps on [0,1]. They have to be uniformly distributed.
The random numbers may be discrete instead of continuous. Each Ri is discrete.
Solution: a very large integer for modulus m
The mean of generated numbers might be too high or too low.
The variance of generated numbers might be too high or too low.
Maximum density should be achieved and cycling should be avoided.

6. Characteristics of a Good Generator (contd.)
Speed and efficiency are aided by a modulus, m, to be (or close to) a power of 2.
For the above point, Maximum Period is very important.
Achieved by: proper choice of a, c, m, and X0.
If m is very large, m = 231 – 1 or m = 248
Dependence can exist :
Autocorrelation b/w numbers
Numbers successively higher or lower than adjacent numbers.
Several numbers above the mean followed by several numbers below the mean

7. Techniques for Generating Random Numbers
Linear Congruential Method (LCM). (most widely used)
Combined Linear Congruential Generators (CLCG).
Random-Number Streams.

8. Linear Congruential Method [Techniques]
To produce a sequence of integers, X1, X2, … between 0 and m-1 by following a recursive relationship:
The selection of the values for a, c, m, and X0 (seed) drastically affects the statistical properties and the cycle length.
If c = 0, the form is multiplicative congruential method
If c != 0, the form is mixed congruential method
The random integers are being generated [0,m-1], and to convert the integers to random numbers:
Ri = Xi / m (I = ,1,2 ….)

9. Example Use X0 = 27, a = 17, c = 43, and m = 100.
The Xi and Ri values are:
X1 = (17*27+43) mod 100
= 502 mod 100
= 2,
R1 = 0.02;
X2 = (17*2+32) mod 100
= 77
R2 = 0.77;
X3 = (17*77+32) mod 100
= 52
R3 = 0.52;

10. Combined Linear Congruential Generators
Reason: Longer period generator is needed because of the increasing complexity of stimulated systems.
Approach: Combine two or more multiplicative congruential generators.
L’Ecuyer suggests that :
If Wi,1, Wi,2, Wi,3 …. Wi,k are any independent, discrete valued variables, Wi,k is uniformly distributed on the integer from 0 to m1-2, then,
Wi = [ Wi,j ] mod m1 -1

11. Let Xi,1, Xi,2, …, Xi,k, be the ith output from k different multiplicative congruential generators
The jth generator:
Has prime modulus mj & multiplier aj is chosen such that period is mj-1
Produces integers Xi,j is approx ~ Uniform on integers in [1,mj-1]
Wi,j = Xi,j -1 is approx ~ Uniform on integers in [1, mj-2]

12. Combined Linear Congruential Generators
The combined generators have periods as long as 2191 approx = 3 x 1057

14. Random Number Streams

15. Tests for Random Numbers
The desirable properties of random numbers is uniformity and independence.
Tests are performed to check whether the two properties are satisfied by the sequence of random numbers generated.
Two types of tests are performed
Frequency test
Autocorrelation test

16. Tests for Random Numbers (contd.)
Testing for uniformity:
H0: Ri ~ U[0,1]
H1: Ri !~ U[0,1]
The null hypothesis H0, reads that the numbers are distributed uniformly on the interval [0,1]
Failure to reject the null hypothesis, H0, means that evidence of non-uniformity has not been detected by this test.
Testing for independence:
H0: Ri ~ independently
H1: Ri !~ independently
This null hypothesis H0, reads that the numbers are independent.
Failure to reject the null hypothesis, H0, means that evidence of dependence has not been detected by this test.

17. Testing for the above to properties and proving that the hypothesis is not rejected does not mean that further tests for the generator are not required.
For each test, a level of significance is a stated.
Level of significance α, the probability of rejecting H0 when it is true:
α = P(reject H0|H0 is true)
The decision maker sets the value of a for any test
Frequently a is set to 0.01 or 0.05

18. If a = 0.05 and 5 different tests are conducted on a set of numbers, then, the probability of rejecting the null hypothesis on atleast one test, by chance alone, could be as large as 0.25
If 100 numbers were subject to a test, with a = 0.05, it would be expected that 5 of those tests would be rejected.
If the number of rejections in 100 tests is close to 100a then, the generator should not be rejected.

19. When to use these tests:
If a well-known simulation language or random-number generator is used, it is probably unnecessary to test
When not to use these tests:
If the generator is not explicitly known or documented
When spreadsheet programs, symbolic/numerical calculators are used,
When random number generators are added to software which are not developed for simulation,

20. Types of tests:
Theoretical tests: evaluate the choices of m, a, and c without actually generating any numbers
Empirical tests: applied to actual sequences of numbers produced, which is our emphasis in this learning.

21. Frequency Tests Test of uniformity Two different methods:
Kolmogorov-Smirnov test
Chi-square test
Both these tests measure the degree of agreement b/w the distribution of a sample of generated random nos. and the theoretical uniform distribution.
Both tests are based on the null hypothesis of no significant difference b/w the sample distribution theoretical distribution.

22. Kolmogorov-Smirnov Test
Compares the continuous cdf, F(x), of the uniform distribution with the empirical cdf, SN(x), of the N sample observations.
We know that : F(x) = x, <= x <= 1
If the sample from the RN generator is
R1, R2, …, Rn,
then the empirical cdf, SN(x) is:
As N becomes larger, SN(x) should become a better approximation to F(x), provided that the null hypothesis is true.

23. It is based on the largest absolute deviation between SN(x) and F(x) over the range of the random variable.
It is based on the statistics :
D = max | F(x) - SN(x) |
Sampling distribution of D is known (a function of N, tabulated in Table A.8.)

24. For performing Kolmogorov test against a uniform cdf, follow the steps :
Rank the data from the smallest to largest
Compute
D+ = max { i/N – Ri }
D- = max { Ri – (i-1)/N }
Compute D = max (D+ , D- )
Locate the critical value of Da for the significant level a and the given sample N in table A.8
If D > Da , then, the null hypothesis H0 is rejected.
If D <= Da , then, there is no difference b/w the true distribution of random numbers generated and the uniform distribution. Hence the null hypothesis is not rejected.

27. The Chi-Square Test N is the total number of observations.
Oi is the observed number in the ith class,
n is the number of classes,
Ei is the expected number in each class,
given by : Ei = N/n for equally spaced classes.
It can be shown that the sampling distribution X2 is approximately the chi-square distribution with n-1 degrees of freedom.

28. Tests for Autocorrelation
The tests for autocorrelation are concerned with the dependence b/w numbers in a sequence.
Consider the sequence of numbers
It appears that every 5th, 10th, 15th number is a large no.
20 – 30 nos. are too small to reject a random no generator, but the notion is that the numbers might be related.
The autocorrelation ρim of interest between numbers :
Ri, Ri+m, Ri+2m,…., Ri+(M+1)m
M is the largest integer such that
i + (M + 1)m <= N,
where N is the total number of values in that sequence.

29. If the values are uncorrelated:
Hypothesis:
H0: ρim = 0
H1: ρim != 0
If the values are uncorrelated:
For large values of M, the distribution of the estimator of ρim, denoted by (cap)ρim is approximately normal, if, Ri, Ri+m, Ri+2m,…., Ri+(M+1)m are uncorrelated.
The test statistics is :
Z0 is distributed normally with mean = 0 and variance =1

30. The formula for is in slightly a different form
The formula for is in slightly a different form. And the standard deviation of the estimator, given by Schmidt and Taylor are as follows :
After computing Z0, do not reject the null hypothesis of independence, if,
-Za/2 <= Z0 <= Za/2
Where a is the level of significance and is obtained from the table A.3. The fig 7.3 illustrates this test.