MONTE CARLO METHOD DISCRETE SIMULATION RANDOM NUMBER GENERATION Chapter 3 : Random Number Generation.

Slides:



Advertisements
Similar presentations
RSLAB-NTU Lab for Remote Sensing Hydrology and Spatial Modeling 1 An Introduction to R Pseudo Random Number Generation (PRNG) Prof. Ke-Sheng Cheng Dept.
Advertisements

Biomedical Statistics Testing for Normality and Symmetry Teacher:Jang-Zern Tsai ( 蔡章仁 ) Student: 邱瑋國.
1 Chi-Square Test -- X 2 Test of Goodness of Fit.
Generating Random Numbers
Chapter 6 Sampling and Sampling Distributions
Hypothesis Testing Steps in Hypothesis Testing:
Random Number Generation. Random Number Generators Without random numbers, we cannot do Stochastic Simulation Most computer languages have a subroutine,
Outline input analysis input analyzer of ARENA parameter estimation
Theoretical Program Checking Greg Bronevetsky. Background The field of Program Checking is about 13 years old. Pioneered by Manuel Blum, Hal Wasserman,
Probability Densities
Point estimation, interval estimation
CF-3 Bank Hapoalim Jun-2001 Zvi Wiener Computational Finance.
Chapter 7 Sampling and Sampling Distributions
Simulation Modeling and Analysis Session 12 Comparing Alternative System Designs.
Statistics.
Random Number Generation
Analysis of Simulation Input.. Simulation Machine n Simulation can be considered as an Engine with input and output as follows: Simulation Engine Input.
Continuous Random Variables and Probability Distributions
Experimental Evaluation
1 Simulation Modeling and Analysis Pseudo-Random Numbers.
Properties of Random Numbers
Copyright (c) 2004 Brooks/Cole, a division of Thomson Learning, Inc. Chapter 7 Statistical Intervals Based on a Single Sample.
Random-Number Generation. 2 Properties of Random Numbers Random Number, R i, must be independently drawn from a uniform distribution with pdf: Two important.
Fall 2011 CSC 446/546 Part 6: Random Number Generation.
ETM 607 – Random Number and Random Variates
Chapter 13: Inference in Regression
Copyright © 2013, 2010 and 2007 Pearson Education, Inc. Chapter Inference on the Least-Squares Regression Model and Multiple Regression 14.
Input Analysis 1.  Initial steps of the simulation study have been completed.  Through a verbal description and/or flow chart of the system operation.
Today’s lesson Confidence intervals for the expected value of a random variable. Determining the sample size needed to have a specified probability of.
Inferences in Regression and Correlation Analysis Ayona Chatterjee Spring 2008 Math 4803/5803.
Mid-Term Review Final Review Statistical for Business (1)(2)
Statistical Decision Making. Almost all problems in statistics can be formulated as a problem of making a decision. That is given some data observed from.
1 Statistical Distribution Fitting Dr. Jason Merrick.
CPSC 531: RN Generation1 CPSC 531:Random-Number Generation Instructor: Anirban Mahanti Office: ICT Class Location:
Chapter 7 Random-Number Generation
Tests for Random Numbers Dr. Akram Ibrahim Aly Lecture (9)
CS433: Modeling and Simulation Dr. Anis Koubâa Al-Imam Mohammad bin Saud University 15 October 2010 Lecture 05: Statistical Analysis Tools.
Basic Concepts in Number Theory Background for Random Number Generation 1.For any pair of integers n and m, m  0, there exists a unique pair of integers.
Discrete Random Variables. Numerical Outcomes Consider associating a numerical value with each sample point in a sample space. (1,1) (1,2) (1,3) (1,4)
Y X 0 X and Y are not perfectly correlated. However, there is on average a positive relationship between Y and X X1X1 X2X2.
Confidence intervals and hypothesis testing Petter Mostad
Chapter 5 Parameter estimation. What is sample inference? Distinguish between managerial & financial accounting. Understand how managers can use accounting.
Selecting Input Probability Distribution. Simulation Machine Simulation can be considered as an Engine with input and output as follows: Simulation Engine.
Tests of Random Number Generators
Chapter 10 Verification and Validation of Simulation Models
Random Variable The outcome of an experiment need not be a number, for example, the outcome when a coin is tossed can be 'heads' or 'tails'. However, we.
Sampling and estimation Petter Mostad
Probability and Distributions. Deterministic vs. Random Processes In deterministic processes, the outcome can be predicted exactly in advance Eg. Force.
Probability Theory Modelling random phenomena. Permutations the number of ways that you can order n objects is: n! = n(n-1)(n-2)(n-3)…(3)(2)(1) Definition:
Continuous Random Variables and Probability Distributions
Significance Tests for Regression Analysis. A. Testing the Significance of Regression Models The first important significance test is for the regression.
G. Cowan Lectures on Statistical Data Analysis Lecture 5 page 1 Statistical Data Analysis: Lecture 5 1Probability, Bayes’ theorem 2Random variables and.
0 Simulation Modeling and Analysis: Input Analysis 7 Random Numbers Ref: Law & Kelton, Chapter 7.
1.  How does the computer generate observations from various distributions specified after input analysis?  There are two main components to the generation.
Statistical principles: the normal distribution and methods of testing Or, “Explaining the arrangement of things”
Statistical Decision Making. Almost all problems in statistics can be formulated as a problem of making a decision. That is given some data observed from.
Random number generation
Chapter 12 Chi-Square Tests and Nonparametric Tests
Active Learning Lecture Slides
Chapter 10 Verification and Validation of Simulation Models
Goodness of Fit Tests The goal of χ2 goodness of fit tests is to test is the data comes from a certain distribution. There are various situations to which.
Chapter 7 Random Number Generation
Chapter 7 Random-Number Generation
Properties of Random Numbers
Discrete Event Simulation - 4
Discrete Event Simulation - 5
Validating a Random Number Generator
Computer Simulation Techniques Generating Pseudo-Random Numbers
Random Number Generation
Skills 5. Skills 5 Standard deviation What is it used for? This statistical test is used for measuring the degree of dispersion. It is another way.
Presentation transcript:

MONTE CARLO METHOD DISCRETE SIMULATION RANDOM NUMBER GENERATION Chapter 3 : Random Number Generation

Monte Carlo method  Solving problems using random numbers  Ex: Sqrt(3)  3 is between the squares (of adjacent numbers) 1 and 4  Sqrt(3) is between the roots 1,2  Monte Carlo principle is to generate N random numbers and count are in (1,  3) 1  3 2

Monte Carlo method  If numbers generated are random then the numbers in any interval is proportional to length of the interval  n/N = (  3-1)/(2-1)   3 = 1+n/N 1  3 2

Monte Carlo method Algorithm sqrtof3 1. Input N 2. Initialize n=0, count=0 3. Generate random number r in (1,2) 4. If r*r <= 3 then 5. n=n+1, endif 6. count=count+1 7. If count < N then 8. Go to 3; else 9. Output “sqrt of 3 is”, 1+n/N

Monte Carlo method  If sqrt of 5 is required then generate numbers in (2,3)  Because 5 in between the squares (4, 9)  Sqrt(5) = 2+n/N  Gordon discusses integration of functions using Monte Carlo method – Go through this

Monte Carlo method  Finding   Consider unit circle. Area is   Enclose a sector in I quadrant in the unit square  Generate N points in the unit square and count how many (n) in the sector of the circle  n/N =  /4. That is  = 4n/N

Monte Carlo method Algorithm phi 1. Input N 2. Initialize n=0, count=0 3. Generate random number x in (0,1) 4. Generate random number y in (0,1) 5. If x*x+y*y <=1 then 6. n=n+1, endif 7. count=count+1 8. If count < N then 9. Go to 3; else 10. Output “value of phi is”, 4n/N

Discrete simulation  Examples  Queue: Here the state changes at random times  Change in number in the system only when an arrival or departure happens  The state need not be updated at intervals of  t  Find when an event happens and change the clock to that time and update state  Also called event to event or next event simulation

Discrete simulation Read input data Generate and store random data and initialize Find next potential event and its time Update clock and state Collect statistics End of simula tion? Print statistics y A A n

Continuous simulation Read input data Generate and store random data and initialize t=0 t=t+dt Update state Collect statistics End of simula tion? Print statistics y A A n

Discrete simulation  In all discrete simulations state changes happen at random times  We learn to generate random numbers  Methods  Use a table  Wheel  Algorithm  Pseudo random numbers generated  So test these for randomness

Discrete simulation  The distribution may be any thing  But all distributions are generated using unit uniform random numbers  Properties:  Uniform  Random  Independent  Why pseudo?  Generated using formula – no real randomness  Apparently random  How to trust?  Perform tests

Pseudo random numbers  Linear congruential generator  c(i+1) = (c(i)*a+b)mod m  Say c(0)=3, a=5, b=2, m=10  We get successive numbers to be …  One other property of generated random numbers  Long sequence  Examples  c(0)=2, a=4, b=6, m=10 – bad example  c(0)=2, a=3, b=0, m=5 – good example

Choice of parameters  Large p – can increase the range – not necessarily  Right choices for a, p, b, c(0) can large set of numbers  Suggestions  Generator is multiplicative  c(0) is odd  p=2 r  a= k*8  3  In practical implementations  p is one more than the largest integer that can fit in a word  = 2 31  Division of a*c(i) is not necessary  As soon as product is > p, the rest of the bits are dropped  Thus the remainder is answer of the product  Ex: 4 bit word – 5 x 7 = 0101 x 0111 = mod 16 = 0011  c(0) is relatively prime to p - any odd number is chosen for p  a= k*8  3

Algorithm Algorithm uun(c0 ) 1. global integer a=189277, p=2 31, c0 2. local integer n 3. c0=a*c0 4. n=c0 mod m 5. n = float (n) * E-9 //divide by return n End uun

Example  p=10 2, a=19, c(0)=63  The product of 19 and 63 is 1197  Just pick least significant digits for remainder  Successive numbers (integers) are 97, 43, 17  These are divided by p to get numbers in (0,1)  So the generated numbers are.97,.43,.17

Parameter selection  p=2 31  a=1+4k / 5+8k / 3+8k  p prime  a k -1 is divisible by p

Combined linear generators  Advantage is longer period  (p 1 -1)(p 2 -1) for combining numbers from two generators  Take i th numbers from all generators and add these (find weighted sum) and find modulo p 1 or p 2 or …

Requirements of algorithm  Fast  Portable  Long cycle  Replicable  Should satisfy all properties that real random numbers do

Tests  Uniformity - frequency test  Randomness test - variety of runs tests  Autocorrelation test - to ensure absence of correlation  Gap test – occurrence of a digit  Independence – Pokers

Uniformity test – Chi square test  Frequency tests  Chi square  Kolmogorov –Smirnov  We set up a null hypothesis that random numbers generated by the pseudo generator are uniform  Chi square test – compare the observed frequency with expected frequency  For example the observed frequencies of 3 digit random numbers in generation of 100 random numbers are Interval O.F E.F (OF-EF) total =62 This  [(OF-EF) 2 /EF] is a random variable with chi square distribution and degrees of freedom 5-1

Chi square test  Calculated chi square value 62/20 = 3.1  Compare this with table value  One sided test: If calculated value is less than the table value accept it to be good generator (accept the null hypothesis)  Even at 90% we accept the generator to be good. (3.1 < 7.78)  Two sided test:  Table values for 4 degrees of freedom  At 95%, the interval of acceptance is (.484, 11.14) and the calculated value 3.71 lies in this interval. Hence accept the random numbers to be uniform  At 99% the interval is (.206, 14.96)

Uniformity test – Kolmogorov-Smirnov test  Kolmogorov-Smirnov  We find cumulative probabilities of the supposedly uniform distribution S N (x)  For uniform random variate F(x) =x, for 0 ≤ x ≤ 1  Assume the sample random numbers from the generator are R 1, R 2, … R N  S N (x) =[number of R 1, R 2, … R N ≤ x]/N  As N becomes larger, S N (x) should become F(x) if the generator generates uniformly distributed numbers  Kolmogorov-Smirnov test is based on the largest absolute deviation between F(x) and S N (x)  That is, calculate max {abs(F(x) - S N (x))}

Kolmogorov-Smirnov test  Kolmogorov-Smirnov  We find cumulative probabilities of the supposedly uniform distribution S N (x)  For uniform random variate F(x) =x, for 0 ≤ x ≤ 1  Assume the sample random numbers from the generator are R 1, R 2, … R N  S N (x) =[number of R 1, R 2, … R N ≤ x]/N  As N becomes larger, S N (x) should become F(x) if the generator generates uniformly distributed numbers  Kolmogorov-Smirnov test is based on the largest absolute deviation between F(x) and S N (x)  That is, calculate max {abs(F(x) - S N (x))}

Kolmogorov-Smirnov test 1. Rank the data R(1)≤ R(2) ≤ …. ≤ R(N) 2. Find D + = max{i/N – R(i)}, D - =max {R(i) –(i-1)/N} 3. Find D= max {D +, D - } 4. Find the critical value C from the table for N and α 5. If the calculated value is > than C, reject null hypothesis that the numbers are uniform; else accept the distribution of numbers to be uniform

Kolmogorov-Smirnov test , 0.14, 0.44, 0.81, Find D + = max{i/N – R(i)}= 0.26, D - =max {R(i) –(i- 1)/N} = Find D= max {D +, D - }= Find the critical value C = from the table for N=5 and α= The calculated value is < than C, accept the distribution of numbers to be uniform R(i) i/N (i-1)/N R(i)-(i-1)/N i/N-R(i)

Runs Tests – number of runs  Basically tests randomness and independence  Example numbers from a generator  The numbers above may pass any of the uniformity tests discussed previously  But these cannot be accepted to be random and independent – Why?  Runs test examines the arrangement of numbers in a sequence to test independence

Runs Tests – number of runs  Let a be random variable denoting number of runs in a sequence of N numbers  Distribution of a is normal with mean (2N-1)/3 and standard deviation (16N-29)/90 (for N > 20)  Thus (a-mean) / s.d is standard normal  We get acceptable range from standard normal table

Runs Tests to test number of runs  Run is a succession of similar order in the sequence  Example:  Find number of runs by finding + – sequence (runs up and down)  There is decrease from 0.87 to 0.15 and hence the first sign is –  There is increase from 0.15 to 0.23 and hence the second sign in the sequence is +  The number of runs is 8  This is the number of succession of – or +  Too many runs or too little runs is unacceptable  In the example of 30 numbers in previous slide the number of runs is 5

Runs Tests to test number of runs  Say 30 random numbers (two digit) and let us test this for number of runs  Also verify that the 30 numbers in slide 27 do not pass this runs test (Is it so?)

Runs Tests to length of runs (up and down)  Consider the example random numbers from a generator  The sequence above will pass frequency test as well as number of runs test (number of runs=6)  What is wrong with the sequence?

Runs Tests to length of runs  The length of runs is too long and not acceptable  The number of + and – in sequence is length  Make a frequency table of this length  Data about the distribution of length of runs of uniform random numbers  E(y i )= (2/(i+3)!)[N(i 2 +3i+1)-(i 3 +3i 2 -i-4)], i≤N-2 =2/N!, i=N-1  Find the O i the observed frequency and do chi square test

Runs Tests to test length of runs  Say 30 random numbers (two digit) and let us test this for the length of runs  If expected frequencies are too small merge two or more classes and find chi square  Also verify that the 30 numbers in slide 31 or 27 do not pass this runs test (Is it so?)

Test length of runs above and below mean  Consider the 30 random numbers (two digit)  This sequence may pass frequency, number of runs and length of runs up and down tests  Then what is the problem with this sequence?

Runs Tests to length of runs (above and below mean)  The length of runs is too long and not acceptable  The number of + and – in sequence is length  Make a frequency table of this length  Data about the distribution of length of runs (above and below mean) of uniform random numbers  E(y i )= (Nw i )/E(I) Where w i = (n1/N) i (n2/N)+(n2/N) i (n1/N) and E(I) = (n1/n2)+(n2/n1)  Find the O i the observed frequency and do chi square test

Test length of runs above and below mean Runs  Say 30 random numbers (two digit) and let us test this for the length of runs  If expected frequencies are too small merge two or more classes and find chi square  Also verify that the 30 numbers in slide 31 or 27 do not pass this runs test

Test length of runs (above and below mean)  Say 30 random numbers (two digit) and let us test this for the length of runs  If expected frequencies are too small merge two or more classes and find chi square  Also verify that the 30 numbers in slide 34 or 31 or 27 do not pass this runs test

Autocorrelation Test  Numbers generated are correlated  A test is designed to make sure the generated numbers simulate absence of correlation  Example random numbers  These numbers is likely to pass all tests discussed so far  Do you see any problem with this sequence?

Autocorrelation Test  After every 5 numbers a high number is found  There is correlation present between numbers placed in positions 5, 10, 15, 20 …  R(i), R(i+m), R(i+2m), … R(i+(M+1)m) be the numbers beginning at i and after every m numbers there onwards  Distribution:  ρ/σ is standard normal  ρ is auto correlation computed for the sequence above and σ is the s.d of auto correlation coefficient  ρ is found as [1/(M+1){  R(i+km)R(i+(k+1)m)}]-0.25  σ is [sqrt(13M+7)]/(12M+12)  Compute this ρ/σ and compare with table values of standard table values

Auto correlation test  In the sequence above do auto correlation test on 3 rd, 8 th, 13 th … numbers  In the sequence above perform the same test on 5 th, 10 th, 15 th, … numbers

Gap test  This is used to determine the significance of the interval between the recurrence of the same digit  List out the digits in particular position  Examine the gap between occurrences of a digit  Something like runs test  Suppose the digits are   The gap lengths in occurrence of digit 3 are 10, 7, 0, 2, 3 Frequency of gaps are of interest  P(gap of 10) = ( )0.1  P(gap of x)=(0.9 x )0.1  P(gap ≤x) = x+1 This is theoretical cumulative frequency  Perform a Kolmogorov-Smirnov test using actual cumulative frequency

Gap test  Can test on the length of gaps  Total number of gaps is N-10  Example of 30 numbers Digit No. of gaps  Make a frequency distribution of gap lengths  Digit Gaps Digit Gaps Digit Gaps 0 none 3 10,7,0,2, ,7,6 4 17,7 7 4,5 2 1,6, /9 19 /5,5

Gap test Gap lengths o.f c.f (S N (x)) F(x) abs(F(x)-S N (x) / / / / Calculated statistic is Table value is 0.24 Hence accept the hypothesis of independence

Poker test  Based on the frequency with which certain digits are repeated  Suppose we generate 3 digit numbers  The numbers can be grouped based on the repetition of digits thus:  All digits different, All digits same, Any two digits same  For example among 0.286, 0.444, 0.445, 0.353,  First number belongs to class 1, second number to class 2, and remaining to class 3  Probability of the 3 classes are 10x9x8/1000 = 0.72, 10/1000=0.01 and 1-( ) = 0.27  If we generate numbers expected numbers in the three classes are 0.72 x = 7200, 100, 2700  Compare this with actual frequencies and perform a chi square test

Summary and references  Monte Carlo method – G.Gordon, Chapter 6, 6.4 N.Deo, Chapter 3, 3.5  General continuous and discrete simulation – N.Deo, Chapter3, 3.1  Congruential generator - N.Deo, Chapter3, 3.3 G.Gordon, Chapter 6, 6.5, 6.6  Tests on random numbers – J. Banks et al., Chapter 7, 7.4