1. General Principle of Monte Carlo Fall 2013 By Yaohang Li, Ph.D.

2. Review Last Class –Short Resume of Statistics Probability Random Variables Important Distributions –Uniform Distribution –Exponential Distribution –Binomial Distribution –Poison Distribution –Normal Distribution Estimation –Sample –Estimand –Parent Distribution –Sampling Distribution –Estimator »Important estimators This Class –Nature of Monte Carlo –Direct Simulation –Monte Carlo Integration Next Class –Analysis of Monte Carlo Integration

3. Mathematical Theory and Experiment Theoretical Mathematics –Deduce conclusions from postulates Experimental Mathematics –Infer conclusions from observations –Complicated Mathematical Experiments require computers Difference –Between deduction and induction

4. Problems Handled by Monte Carlo Probabilistic Problems –Directly concern with the behavior and outcome of random processes –Approach (Direct Simulation) Choose random numbers in a certain way Directly simulate the physical random process of the original problem Infer the desired solution from the behavior of these random numbers –Usually possess little theoretical interest Deterministic –Not directly concern with the behavior of random processes –Usually called Sophisticated Monte Carlo

5. Strength and Weakness of Theoretical Mathematics Theoretical Mathematics –Strength Abstraction and Generality Abstract the essence of a problem and reveal its underlying structure –Inherent weakness The more general and formal the theory The harder to provide a numerical solution in a particular application Main idea behind Monte Carlo approach to deterministic problems –Exploit the strength of theoretical mathematics –Avoid its associated weakness by replacing theory by experiment whenever the theory falters

6. Example of Sophisticated Monte Carlo Electromagnetic Theory –Requires solution of Laplace’s equation subject to certain boundary conditions –The boundary conditions may be complicated and no analytical techniques are available Monte Carlo Method –Study of particles Diffuse randomly in a region bounded by absorbing barriers Guide the particles by means of random numbers Until they are absorbed on barriers –Barriers are chosen to represent the prescribed boundary conditions –The distribution of particles absorbed on barriers gives the solution

7. Monte Carlo Solution –Uncertain Arise from raw observational data consisting of random numbers If we can manage to make the uncertainty fairly negligible, Monte Carlo can serve a useful purpose –Good experiments Ensure the sample should be more rather than less representative in the Monte Carlo computation –Good representation of conclusions Indicates how likely the solution can be wrong

8. Reducing Uncertainty –Collect more observations Not very economic –Variance Reduction Manipulate the mathematical problem Reduce the inherent variance

9. Numerical Integration Methods for approximating definite integrals –Rectangle Rule –Trapezoidal Rule Divide the curve into N strips of thickness h=(b-a)/N Sum the area of each trip –Approximate to that of a trapezium –Simpson’s Rule Calculate quadratic instead

10. Monte Carlo Method General Principles –Every Monte Carlo computation that leads to quantitative results may be regarded as estimating the value of a multiple integral Efficiency –Definition Suppose there are two Monte Carlo methods Method 1: n1 units of computing time,  1 2 Method 2: n2 units of computing time,  2 2 Methods comparison

11. Monte Carlo Integration Consider a simple integral Definition of expectation of a function on random variable  If  is uniformly distributed, then

12. Crude Monte Carlo –If  1, …,  n are independent random numbers Uniformly distributed –then fi=f(  i) are random variates with expectation  –is an unbiased estimator of  –The variance is –The standard error is  /n 1/2

13. Hit-or-Miss Monte Carlo Revisit Suppose 0  f(x)  1 when 0  x  1 Main idea –draw a curve y=f(x) in the unit square 0  x,y  1 –is the proportion of the are of the square beneath the curve –or we can write

14. Analysis of Hit-or-Miss Monte Carlo  can be estimated as the a double integral The estimator of hit-or-miss Monte Carlo

15. Hit-or-Miss Monte Carlo –We take n points at random in the unit square, and count the proportion of them which lie below the curve y=f(x) –The points are either in or out of the area below the curve The probability that a point lies under the curve is  The Hit-or-Miss Monte Carlo is a Bernoulli trial –the estimator of Hit-or-Miss Monte Carlo is binomial distributed

16. Binomial Distribution Revisit Binomial Distribution –Discrete probability distribution Pp(n|N) of obtaining exactly n successes out of N Bernoulli trials –Each Bernoulli trial is true with probability p and false with probability q=1-p variance N(1-p)p = =

17. Comparison of Hit-or-Miss Monte Carlo and Crude Monte Carlo Standard error of Hit-or-Miss Monte Carlo Standard error of Crude Monte Carlo Hit-or-Miss Monte Carlo is always worse than Crude Monte Carlo –Why?

18. Why Hit-or-Miss Monte Carlo is worse? Fact –The hit-or-miss to crude sampling is equivalent to replacing g(x,  ) by its expectation f(x) –The y variable in g(x,y) is a random variable Leads uncertainty Leads extra uncertainty in the final results Can be replace by exact value

19. General Principle of Monte Carlo If, at any point of a Monte Carlo calculation, we can replace an estimate by an exact value, we shall replace an estimate by an exact value, we shall reduce the sampling error in the final result

20. Summary Nature of Monte Carlo –Experimental Mathematics and Theoretical Mathematics –Problems that Monte Carlo can handle Probabilistic –Direct Simulation Deterministic –Sophisticated Monte Carlo –Monte Carlo Solution Good Experiment Good Conclusion Monte Carlo Integration –Hit or Miss Monte Carlo –Crude Monte Carlo

21. What I want you to do? Review Slides Review basic probability/statistics concepts Work on your Assignment 1