1. Computer Simulation Techniques Generating Pseudo-Random Numbers
Lebanese American University
Rayana H. Jaafar

2. Random Numbers Applications
Fields of Application
Numerical Analysis
Computer Programming
Cryptography
Simulation
Model Randomness

3. Random Numbers in Simulation
Machine Interface Model
Machine Operation Time
Machine Repair Time
Realistic Simulations Models
Generate Random Numbers that follow a given Theoretical or Empirical Distribution

4. Pseudo-Random Numbers
Pseudo-Random Numbers (RN)
Uniform Distribution
Range ∈[0,1]
Stochastic or Random Variates
Other Theoretical or Empirical Distribution

5. Random Numbers Generation
True Random Numbers
Physical Phenomena
Unpredictable & Non-Reproducible Source
Thermal Noise from a Semi-Conductor Diode
Particle Emission in Radioactive Decay
Disadvantages
High Cost of Production and Storage
Non-Reproducible Not Useful for Computer Simulation

6. Random Numbers Generation
Pseudo-Random Numbers
Mathematical Algorithms
Generate Numbers Deterministically
Appear as Random Numbers
Statistical Tests for Randomness
Advantages
Reproducible Useful for Computer Simulation
Generated on Demand

7. Random Numbers Properties
Sequences of Random Numbers or Bits
Uniformly Distributed
Statistically Independent
Reproducible
Non-Repeating for the Desired Length
“The list of widely used generators that should be discarded is long... Check the default PRNG of your favorite software and be ready to replace it if needed. This last recommendation has been made over and over again over the past 40 years. Perhaps amazingly, it remains as relevant today as it was 40 years ago.”
International Encyclopedia of Statistical Science

8. Von Neumann’s Mid-Square Method
First Method to Generate RNs By Computers
Square the previous random number
Extract middle digits
How Can This Method Generate a Sequence of RNs ?

9. The Congruential Method
Based on the following Recursive Algorithm
𝑥 𝑖+1 = 𝑎 𝑥 𝑖 +𝑐 𝑚𝑜𝑑 𝑚
0<𝑎<𝑚, 0≤𝑐<𝑚
𝑥 0 = 𝑎= 𝑐= 𝑚=10 …
𝑇=4
Seed

10. Tausworthe Generators (TG)
Additive Congruential Generators where 𝑚=2
𝑥 𝑖 = 𝑎 1 𝑥 𝑖−1 + …+ 𝑎 𝑛 𝑥 𝑖−𝑛 𝑚𝑜𝑑 2
𝑥 𝑖 = 𝑎 1 𝑥 𝑖−1 ⨁ …⊕ 𝑎 𝑛 𝑥 𝑖−𝑛
𝑥 𝑖 ∈ 0,1 , 𝑎 𝑖 ∈ 0,1
Characteristics
Very Long Cycles
Relatively Slow

11. Lagged Fibonacci Generators (LFG)
General Form LFG
𝑥 𝑛 = (𝑥 𝑛−𝑗 Ο 𝑥 𝑛−𝑘 ) 𝑚𝑜𝑑 𝑚
0<𝑗<𝑘
Characteristics
Produce RNs with Very Good Statistical Properties
Execution can be Parallelized
Highly Sensitive to the Seed

12. 𝑥 𝑘+𝑛 = 𝑥 𝑘+𝑚 ⨁ 𝑥 𝑘 𝑢 | 𝑥 𝑘+1 𝑙 A k≥0
Mersenne Twister (MT)
Generates sequence of bits grouped as blocks
𝑥 𝑘+𝑛 = 𝑥 𝑘+𝑚 ⨁ 𝑥 𝑘 𝑢 | 𝑥 𝑘+1 𝑙 A k≥0
𝐴 is a 𝑤×𝑤 matrix … … 0 ⋮ ⋮ ⋮ ⋱ ⋮ … 1 𝑎 𝑤−1 𝑎 𝑤−2 … … 𝑎 0
𝑥 is a block of 𝑤 bits
𝑛 is the degree of recurrence relation
𝑥 𝑘 𝑢 is the upper 𝑤−𝑟 bits of 𝑥 𝑘
𝑥 𝑘+1 𝑙 is the lower 𝑟 bits of 𝑥 𝑘+1
𝑟 is the separation point
1≤𝑚<𝑛

13. Mersenne Twister The multiplication by 𝐴 can be done as follows
𝑥𝐴= 𝑥≫ 𝑖𝑓 𝑥 0 =0 𝑥≫1 ⊕𝑎 𝑖𝑓 𝑥 0 =1
Where 𝑎= 𝑎 𝑤−1 … 𝑎 0 and 𝑥= 𝑥 𝑤−1 … 𝑥 0
To increase statistical properties, multiply 𝑥 by 𝑇, equivalent to
𝑥 →𝑍=𝑥𝑇
𝑦=𝑥⊕ 𝑥≫𝑞
𝑦=𝑦⨁ 𝑦≪𝑠 ∧𝑏
𝑦=𝑦⨁ 𝑦≪𝑡 ∧𝑐
Z=𝑦⊕ 𝑦≫𝑝

14. Hypothesis Testing Null Hypothesis 𝐻 0 Alternative Hypothesis 𝐻 𝑎
“The produced sequence is random”
Alternative Hypothesis 𝐻 𝑎
“The produced sequence is not random”
For PRNGs, we test the randomness of an output sequence
The result of the statistical test itself is probabilistic
Decision
Real Situation
𝐻 0 Accepted
𝐻 0 Rejected
𝐻 0 True
Valid
Type I Error
𝐻 0 Not True
Type II Error
𝑃𝑟 𝑇𝑦𝑝𝑒 𝐼 =𝛼
Level of Significance

15. Frequency Test
Tests whether the number of 0’s and 1’s is about the same
Steps
Generate 𝑛 Pseudo-Random Numbers to form a sequence
Convert the 0’s to -1’s and add the bits 𝑆 𝑛 = 𝑋 1 + …+ 𝑋 𝑛
Compute 𝑆 𝑜 = 𝑆 𝑛 𝑛
Compute 𝑝=𝑒𝑟𝑓𝑐⁡ 𝑆
If 𝑝<0.01
Reject 𝐻 0
Else the sequence is accepted as random

16. Frequency Test Example
Sequence
Convert to
𝑆 𝑛 =6∗ 1 +4∗ −1 =2
𝑆 𝑜 = 𝑆 𝑛 =0.6324
𝑝= 𝑒𝑟𝑓𝑐 =0.527 >0.01
Sequence is accepted as Random

17. Serial Test
Checks the randomness of overlapping blocks of 𝑘, 𝑘−1, and 𝑘−2 bits found in a sequence
Let 𝑒 be a sequence of 𝑛 bits and 𝑘< 𝑙𝑜𝑔 2 𝑛 −2
𝑒 1 ′ =𝑒|(𝑓𝑖𝑟𝑠𝑡 𝑘−1 𝑏𝑖𝑡𝑠 𝑜𝑓 𝑒)
𝑒 2 ′ =𝑒|(𝑓𝑖𝑟𝑠𝑡 𝑘−2 𝑏𝑖𝑡𝑠 𝑜𝑓 𝑒)
𝑒 3 ′ =𝑒|(𝑓𝑖𝑟𝑠𝑡 𝑘−3 𝑏𝑖𝑡𝑠 𝑜𝑓 𝑒)
Let 𝑓 𝑖 , 𝑓 𝑖 ′ and 𝑓 𝑖 ′′ be the frequency of occurrence of the 𝑖 𝑡ℎ 𝑘, 𝑘−1, and 𝑘−2 overlapping bit combinations in 𝑒 1 ′ , 𝑒 2 ′ and 𝑒 3 ′ respectively.

18. Serial Test Compute the following 𝑆 𝐾 2 = 2 𝑘 𝑛 𝑖 𝑓 𝑖 2 −n
∆𝑆 𝑘 2 = 𝑆 𝐾 2 − 𝑆 𝐾−1 2 and ∆ 2 𝑆 𝑘 2 = 𝑆 𝐾 2 − 2 𝑆 𝐾− 𝑆 𝐾−2 2
𝑃 1 =𝐼𝑛𝑐𝑜𝑚𝑝𝑙𝑒𝑡𝑒𝐺𝑎𝑚𝑚𝑎( 2 𝑘−2 , ∆𝑆 𝑘 2 2 )
𝑃 2 =𝐼𝑛𝑐𝑜𝑚𝑝𝑙𝑒𝑡𝑒𝐺𝑎𝑚𝑚𝑎( 2 𝑘−3 , ∆ 2 𝑆 𝑘 2 2 )
If 𝑃 1 𝑜𝑟 𝑃 2 <0.01
Reject 𝐻 0
Else the sequence is accepted as random

19. Serial Test Example Let 𝑒=0011011101, 𝑛=10 and 𝑘=3 𝑒 1 ′ =001101110100
𝑒 2 ′ =
𝑒 3 ′ =
For 𝑒 1 ′ 𝑘=3 𝑏𝑖𝑡𝑠
𝐶 1 =000, 𝐶 2 =001, 𝐶 3 =010, 𝐶 4 =011, 𝐶 5 =100, 𝐶 6 =101, 𝐶 7 =110, 𝐶 1 =111
𝑓 1 =0, 𝑓 2 =1, 𝑓 3 =1, 𝑓 4 =2, 𝑓 5 =1, 𝑓 6 =2, 𝑓 7 =2, 𝑓 8 =1
For 𝑒 2 ′ 𝑘−1=2 𝑏𝑖𝑡𝑠
𝑓 1 ′ =1, 𝑓 2 ′ =3, 𝑓 3 ′ =3, 𝑓 4 ′ =3
For 𝑒 3 ′ 𝑘−2=1 𝑏𝑖𝑡
𝑓 1 ′′ =4, 𝑓 2 ′′ =6

20. Sequence is accepted as Random
Serial Test Example
𝑆 3 2 =2.8, 𝑆 2 2 =1.2 and 𝑆 1 2 =0.4
∆𝑆 3 2 =1.6 and ∆ 2 𝑆 3 2 =0.8
𝐼𝐺 2,0.8 =0.905>0.01
𝐼𝐺 1,0.4 =0.8805>0.01
Sequence is accepted as Random

21. Autocorrelation Test
Let 𝑒 be a sequence of 𝑛 bits and let 𝑒 𝑖 be 𝑒 shifted by 𝑖 positions
Consider 1≤𝑑≤ 𝑛 2
Find the number of different bits between 𝑒 𝑖 and 𝑒 𝑖+𝑑 as follows
𝐴 𝑑 = 𝑖=0 𝑛−𝑑−1 𝑒 𝑖 ⊕ 𝑒 𝑖+𝑑
Compute 𝑆= 2 𝐴 𝑑 − 𝑛−𝑑 𝑛−𝑑
For 5% level of significance, we accept 𝐻 0 if 𝑆≤1.96

22. Runs Test 𝑅= 1 𝑛 1≤𝑖,𝑗≤6 𝑟 𝑖 −𝑛 𝑝 𝑖 𝑟 𝑖 −𝑛 𝑝 𝑗 𝑎 𝑖𝑗
Let 𝑒 be a sequence of 𝑛 bits and let 𝑟 𝑖 be the nb of run-ups with length 𝑖
All run-ups with 𝑖>6 are grouped together
𝑅= 1 𝑛 1≤𝑖,𝑗≤6 𝑟 𝑖 −𝑛 𝑝 𝑖 𝑟 𝑖 −𝑛 𝑝 𝑗 𝑎 𝑖𝑗
For 𝑛≥4000, 𝑅 has a Chi-Square distribution under the assumption that the random numbers are independent and 𝑖.𝑖.𝑑

23. Chi-Square Test for Goodness of Fit
Let 𝑒 be a sequence of 𝑛 random numbers ∈[0,1]
Divide the interval [0,1] into 𝑘 equal sub-areas
Let 𝑓 𝑖 be the number of random numbers in sub-area 𝑖
If the numbers are truly uniformly distributed, the average number of random numbers in each interval should be 𝑛/𝑘
𝜒 2 = 𝑘 𝑛 𝑖=1 𝑘 𝑓 𝑖 − 𝑛 𝑘 2
Based on the level of significance, and DoF, 𝐻 0 is rejected if 𝜒 2 has a greater value that the one of obtained from Chi-Square table