1. Random-Number Generation Andy Wang CIS 5930-03 Computer Systems Performance Analysis

2. Generate Random Values Two steps –Random-number generation Get a sequence of random numbers distributed uniformly between 0 and 1 –Random-variate generation Transform the sequence to produce random values satisfying the desired distribution 2

3. 3 Background The most common method –Use a recursive function x n = f(x n-1, x n-2, …)

4. Example x n = (5x n-1 + 1) %16 –Suppose x 0 = 5 –The first 32 numbers are between 0 and 15 Divide x n by 15 to get numbers between 0 and 1 4

5. Basic Terms x 0 = seed –Given a function, the entire sequence can be regenerated with x 0 Generated numbers are pseudo random –Deterministic –Can pass statistical tests for randomness –Preferred to fully random numbers so that simulated results can be repeated 5

6. Cycle Length Note that starting with the 17 th number, the sequence repeats –Cycle length of 16 6

7. More Terms Some generators do not repeat the initial part (tail) of the sequence Period of a generator = tail + cycle length 7 tailcycle length period

8. Question How to choose seeds and random- number generation functions? 1.Efficiently computable Heavily used in simulations 2.The period should be large 3.Successive values should be independent and uniformly distributed 8

9. Types of Random-Number Generators Linear-congruential generators Tausworth generators Extended Fibonacci generators Combined generators Others 9

10. 10 Linear-Congruential Generators In 1951, Lehmer found residues of successive powers of a number have good randomness properties x n = a n % m = aa n-1 % m = ax n-1 % m Lehmer’s choices of a and m a = 23 (multiplier) m = 10 8 + 1 (modulus) Implemented on ENIAC

11. (Mixed) Linear-Congruential Generators (LCG) x n = (ax n-1 + b) % m x n is between 0 and m – 1 a and b are non-negative integers “Mixed”  using both multiplication by a and addition by b 11

12. The Choice of a, b, and m m should be large –Period is never longer than m To compute % m efficiently –Make m = 2 k –Just truncate the result by k bits 12

13. The Choice of a, b, and m If b > 0, maximum period m is obtained when –m = 2 k –a = 4c + 1 –b is odd –c, b, and k are positive integers 13

14. Full-Period Generators Generators with maximum possible periods Not equally good –Look for low autocorrelations between successive numbers –x n = ((2 34 + 1)x n-1 + 1) % 2 35 has an autocorrelation of 0.25 –x n = ((2 18 + 1)x n-1 + 1) % 2 35 has an autocorrelation of 2 -18 14

15. Multiplicative LCG x n = ax n-1 % m, b = 0 Can compute more efficiently when m = 2 k However, maximum period is only 2 k-2 Problem: Cyclic patterns with lower bits 15

16. Multiplicative LCG with m = 2 k When a = 8i ± 3 –E.g., x n = 5x n-1 % 2 5 Period is only 8 Which is ¼ of 2 5 When a ≠ 8i ± 3 –E.g., x n = 7x n-1 % 2 5 Period is only 4 16

17. Multiplicative LCG with m ≠ 2 k To get a longer period, use m = prime number –With proper choice of a, it is possible to get a period of m – 1 –a needs to be a prime root of m If and only if a n % m ≠ 1 for n = 1..m - 2 17

18. Multiplicative LCG with m ≠ 2 k x n = 3x n-1 % 31 –x 0 = 1 –Period is 30 –3 is a prime root of 31 18

19. Multiplicative LCG with m ≠ 2 k x n = 7 5 x n-1 % (2 31 – 1) –7 5 is a prime root of 2 31 – 1 –But watch out for computational errors Multiplication overflow –Need to apply tricks mentioned in p. 442 Truncation due to the number of digits available 19

20. 20 Tausworthe Generations How to generate large random numbers? The Tausworthe generator produces a random sequence of binary digits –The generator then divides the sequence into strings of desired lengths –Based on a characteristic polynomial

21. Tausworthe Example Suppose we use the following characteristic polynomial x 7 + x 3 + 1 –The corresponding generation function is b n+7  b n+3  b n = 0 Or b n = b n-4  b n-7 –Need a 7-bit seed 21

22. Tausworthe Example The bit stream sequence 1111111000011101111001011001…. Convert to random numbers between 0 and 1, with 8-bit numbers x 0 = 0.11111110 2 = 0.99219 10 x 1 = 0.00011101 2 = 0.11328 10 x 2 = 0.11100101 2 = 0.89453 10 … 22

23. Tausworthe Generator Characteristics For the L-bit numbers generated +E[x n ] = ½ +V[x n ] = 1/12 +The serial correlation is zero + Good results over the complete cycle - Poor local behavior within a sequence 23

24. Tausworthe Example If a characteristic polynomial of order q has a period of 2 q – 1, it is a primitive polynomial For x 7 + x 3 + 1 q = 7 Sequence repeats after 127 bits = 2 7 - 1 A primitive polynomial 24

25. Tausworthe Implementation Can be easily generated via linear- feedback shift-registers For x 5 + x 3 + 1 25 bnbn b n-1 b n-2 b n-3 b n-4 b n-5 

26. 26 Extended Fibonacci Generators x n = (x n-1 + x n-2 ) % m –Does not have good randomness properties –High serial correlation An extension –x n = (x n-5 + x n-17 ) % 2 k

27. 27 Combined Generations Add random numbers by two or more generators –Can considerably increase the period and randomness x n = 40014x n-1 % 2147483563 y n = 40692y n-1 % 2147483399 w n = (x n - y n ) % 2147483562 –This generator has a period of 2.3 x 10 18

28. 28 w n = 157w n-1 % 32363 x n = 146x n-1 % 31727 y n = 142y n-1 % 31657 v n = (w n - x n + y n ) % 32362 –This generator has a period of 8.1 x 10 12 –Can avoid the multiplication overflow problem Combined Generators

29. 29 XOR random numbers by two or more generators Combined Generators

30. 30 Shuffle –One sequence as an index To an array filled with random numbers generated by the second sequence –The chosen number in the second sequence is replaced by a new random number –Problem Cannot skip to the n th random number Combined Generators

31. 31 A Survey of Random- number Generators Some published generator functions x n = 7 5 x n-1 % (2 31 – 1) –Full period of 2 31 – 2 –Low-order bits are randomly distributed Many others (see textbook) –All have problems General lessons: Use established ones; Do not invent your own

32. 32 Seed Selection If the generator has a full period –Only one random variable is required –Any seed value is good However, with more than one random variable, the story is different for multistream simulations –E.g., random arrival and service times –Should use two streams of random numbers

33. Seed Selection Guidelines Do not use zero –Not good for multiplicative LCGs and Tausworthe generators Avoid even values –Not good if a generator does not have a full period Do not use one stream for all variables –May yield strong correlations among variables 33

34. Seed Selection Guidelines Use nonoverlapping streams –Each stream requires a separate seed –Otherwise… A long interarrival time may correlate with a long service time –Suppose we need 10,000 random numbers for interarrival times; 10,000 for service times, use seeds 1 and 10,001 –x n = [a n x 0 + c(a n – 1)/(a – 1)] % m For multiplicative LCGs, c = 0 34

35. Seed Selection Guidelines Not to reuse seeds in successive simulation runs –No point to run a simulation again with the same seed –Just continue with the last random number as the seed for the successive runs 35

36. Seed Selection Guidelines Do not use random random-number generator seeds –E.g., do not use the time of day, or /dev/random to seed simulations –Simulations should be repeatable –Cannot guarantee that multiple streams will not overlap Do not use numbers generated by random-number generators as seeds 36

37. 37 Myths About Random- number Generation A complex set of operations leads to random results –Hard to guess does not mean random Random numbers are not predictable –Given a few successive numbers from an LCG –Can solve a, c, and m –Not suitable for cryptographic applications

38. 38 Some seeds are better than others –True –Avoid generators whose period and randomness depend on the seed Accurate implementation is not important –Watch out for overflows and truncations Myths about Random- number Generation

39. 39 Bits of successive words generated by a random-number generator are equally randomly distributed –Nope Myths about Random- number Generation

40. 40 x n = (25173x n-1 + 13849) % 2 16 –x 0 = 1 –Least significant bit is always 1 –Bit 2 is always 0 –Bit 3 has a cycle of 2 –Bit 4 has a cycle of 4 –Bit 5 has a cycle of 8 Myths about Random- number Generation ndecimalbinary 1251730110001001010101 2123450011000000111001 3545091101010011101101 4278250110110010110001 5554931101100011000101 6254490110001101101001 7132770011001111011101

41. 41 For all multiplicative LCGs The Lth bit has a period that is at most 2 L For LCGs, with the form x n = ax n-1 % 2 k –The least significant bit is always 0 or always 1 High-order bits are more random Myths about Random- number Generation

42. More on Random Number Generations Mersenne twister –Period =~ 2 19937-1 /dev/random –Extract randomness from physical devices –Truly random 42

43. 43 White Slide