1 Lesson 8: Basic Monte Carlo integration We begin the 2 nd phase of our course: Study of general mathematics of MC We begin the 2 nd phase of our course: Study of general mathematics of MC Consists of a progression: Consists of a progression: Monte Carlo evaluation of integrals (4 ways) Monte Carlo evaluation of integrals (4 ways) Basic numerical analysis framework (to explain the 4 ways) Basic numerical analysis framework (to explain the 4 ways) MC evaluation of integral equations MC evaluation of integral equations Generalization of this technique to solve general differential equation sets Generalization of this technique to solve general differential equation sets

2 Monte Carlo Integration Next set of mathematical tools: MC integration Next set of mathematical tools: MC integration Our study so far of sampling from distributions has provided us with the tools for MC simulation Our study so far of sampling from distributions has provided us with the tools for MC simulation MC integration will provide: MC integration will provide: More rigorous ideas of keeping score More rigorous ideas of keeping score Basic mathematical underpinnings of variance reduction. Basic mathematical underpinnings of variance reduction. “Abstract” approach to MC problem: ALMOST ALL MC PROBLEMS ARE INTEGRATIONS “Abstract” approach to MC problem: ALMOST ALL MC PROBLEMS ARE INTEGRATIONS Development of four particular methods using the framework. Development of four particular methods using the framework.

3 Four particular integration methods We will now go over four particular variations on this theme: We will now go over four particular variations on this theme: 1. Rejection method 2. Averaging method 3. Control variates method 4. Importance sampling method

4 Rejection method This is a similar approach to the use of rejection methods in picking from a distribution. This is a similar approach to the use of rejection methods in picking from a distribution. It is a "dart board" method in which we estimate the area under a functional curve by containing the curve in a rectangular "box", picking a point randomly in the box, and scoring 0 if it misses (i.e., is above the curve) or the full rectangular area if it hits (i.e., is below the curve). It is a "dart board" method in which we estimate the area under a functional curve by containing the curve in a rectangular "box", picking a point randomly in the box, and scoring 0 if it misses (i.e., is above the curve) or the full rectangular area if it hits (i.e., is below the curve). As before, we have to specify an upper bound of the function,, and then proceed by: As before, we have to specify an upper bound of the function,, and then proceed by:

5 Rejection method (2) 1. Choose a value of uniformly between a and b. 2. Choose a value of uniformly between 0 and 3. Score if and score otherwise. and score otherwise.

6 Rejection method example  Find using a rejection method.  Answer: The maximum value of this function in the range is 4, so our procedure is: 1. Choose a value of uniformly between 0 and Choose a value of uniformly between 0 and Score 8 if is less than ; otherwise score 0.  Find first two moments of this method and calculate the expected mean and SD of mean.

7 Averaging method  This is a much more straight-forward approach to the problem because it uses the function directly. The procedure for this method is to: 1. Choose a value of uniformly between a and b. 2. Score

8 Averaging Example  Again find using an averaging method.  Answer: The procedure is to: 1. Choose a value of uniformly between 0 and Score  Find first two moments of this method and calculate the expected mean and SD of mean. (Compare to previous method.)

9 Control variates method  This method is the first of two methods that utilize a user-supplied second function,, which is chosen to be a "well behaved" approximation to  What makes these methods so powerful is that they allow the user to take use of a priori knowledge about the function.  In the control variates method, the integral solution "begins" as the integral of the known function:  and uses the Monte Carlo approach to find an additive correction to this user-supplied guess.

10 Control variates method (2)  The procedure for this method is to: 1. Choose a value of uniformly between a and b. 2. Score  Notice that there is NO variance introduced through the part of the score.  Obviously, then a good guess will result in a small difference and, therefore a small variance. In the limit of a perfect guess,, there is no correction and no therefore no variance. In the limit of a perfect guess,, there is no correction and no therefore no variance. Not quite as obvious is the fact that if h(x) and f(x) differ by a CONSTANT, we also have a 0 variance method. Not quite as obvious is the fact that if h(x) and f(x) differ by a CONSTANT, we also have a 0 variance method.

11 Control variates example  Again find, this time using a control variates method with  Answer: Note the integral of h(x) over (0,2) is 2. With this value known, the procedure is to: 1. Choose a value of uniformly between 0 and Score  Find first two moments of this method and calculate the expected mean and SD of mean. (Compare to previous methods.)

12 Importance sampling method  The final method is the importance sampling method. This technique is similar to the control variates method, in that it takes advantage of a priori knowledge about the function, but differs from it in that its correction is multiplicative rather than additive.  The importance sampling method uses the approximate function as the probability distribution with which the variables are drawn:

13 Importance sampling (2)  The resulting score is:  As with control variates, a "perfect" guess of would result in a zero variance solution, this time because, again, every score would be exactly correct.  (Note that, because of the normalization, a guess equal to a MULTIPLE of f(x) will also work.)

14 Importance sampling example  Again find, this time using an importance sampling method with Answer: Since the integral of h(x) over the range (0,2) is 2, the resulting probability distribution from which to pick the x’s will be: Following the direct procedure for choosing from this distribution, we first determine the c.d.f, which is:

15 Importance sampling example  We then set this c.d.f. to the uniform deviate: and invert to get the formula: Score is now: Find first two moments of this method and calculate the expected mean and SD of mean. (Compare to previous methods.)Find first two moments of this method and calculate the expected mean and SD of mean. (Compare to previous methods.)

16 2 nd pass at integration: more rigor Theoretical underpinning is the Law of Large Numbers Theoretical underpinning is the Law of Large Numbers In one of our early lectures, we defined the mean of a continuous function as: In one of our early lectures, we defined the mean of a continuous function as: And later worked out a Monte Carlo algorithm with the same expectation: And later worked out a Monte Carlo algorithm with the same expectation:

17 Law of Large Numbers (2) Remember that the Law of Large Number takes this a step further by replacing the x with a function f(x) and speaking of the average value of the function, : Remember that the Law of Large Number takes this a step further by replacing the x with a function f(x) and speaking of the average value of the function, : This relates the result of a continuous integration with the result of a discrete sampling. All MC comes from this. This relates the result of a continuous integration with the result of a discrete sampling. All MC comes from this.

18 Using the Law of Large Numbers Putting our “goal” integration in this form requires that we multiply and divide by the probability distribution,  (x) Putting our “goal” integration in this form requires that we multiply and divide by the probability distribution,  (x) Following the previous “rules” we have divided the integrand into two “pieces”: the score and the PDF Following the previous “rules” we have divided the integrand into two “pieces”: the score and the PDF There is an implicit requirement that  (x)>0 for all x for which f(x) is not 0 so that f(x)  (x)/  (x) is defined There is an implicit requirement that  (x)>0 for all x for which f(x) is not 0 so that f(x)  (x)/  (x) is defined

19 Dirac notation In our integrations so far, I have simplified the mathematics a bit by always choosing x between a and b. In our integrations so far, I have simplified the mathematics a bit by always choosing x between a and b. I was careful to always choose x between a and b. What if I had not done this? I was careful to always choose x between a and b. What if I had not done this?

20 Dirac notation (2) A more genearl way to approach this (which takes care of the “domain question”) is to look at the Monte Carlo attack of the integral in TWO steps: (1) an approximation of f(x) itself using: (2) a substitution of this functional approximation into the integral:

21 Dirac notation (3) This is the approach we will take from now on. The notation: has the advantage of giving us not only the “weight” but also reminding us of the selected point. This way we can think of a “sample” as having these two pieces: a “weight” and a “location”

22 Averaging method The easiest of our four methods to put in this form is the averaging method (which we previously discussed second) The easiest of our four methods to put in this form is the averaging method (which we previously discussed second) Recall that the procedure for this method is to: Recall that the procedure for this method is to: Choose a value of uniformly between a and b. Choose a value of uniformly between a and b. Score Score In terms of our mathematical framework, this is equivalent to again using: In terms of our mathematical framework, this is equivalent to again using: and scoring with a direct use of and scoring with a direct use of

23 Averaging Example with Dirac  For the third time, find, this time using Dirac approximation  Answer: The Dirac approximation is:

24 Averaging Example with Dirac (2)  If we use:, then we are guaranteed that, giving us: which is equivalent to the averaging method

25 Averaging Example with Dirac (3)  If we use: then plugging in gives us the importance sampling result: then plugging in gives us the importance sampling result:

26 Rejection method Backing up to the rejection method, the procedure was: 1. Choose a value of uniformly between a and b. 2. Choose a value of uniformly between 0 and 3. Score if and score otherwise. and score otherwise. In terms of our mathematical framework, this is equivalent to using: In terms of our mathematical framework, this is equivalent to using: (for a uniform distribution between a and b) and … (for a uniform distribution between a and b) and …

27 Rejection method (2) scoring with a probability mixing strategy of: with probability with probability or scoring or scoring 0 with probability 0 with probability This mixed scoring strategy obviously has the desired expected value of This mixed scoring strategy obviously has the desired expected value of

28 Control variates method  The procedure for this method is to: 1. Choose a value of uniformly between a and b. 2. Score where, h(x) is chosen as an easily integrated approximation of f(x)+constant where, h(x) is chosen as an easily integrated approximation of f(x)+constant

29 Control variates method (2)  In terms of our mathematical framework, this again uses a flat distribution and score with:

30 Importance sampling method  The procedure for this method is to: 1. Choose a value of between a and b using a probability distribution h(x) that is “shaped like” f(x). 2. Score  In terms of our mathematical framework, this is a simple replacement of the flat distribution of the averaging method with the “better” distribution h(x) (with allowance for the fact that h(x) is probably unnormalized):

31 Importance sampling method(2)  Giving us:

32 Solution of Integral Equations Application of our integration techniques to integral equations Introduction of Dirac notation Introduction of Dirac notation Conversion of differential equations to integral equations Conversion of differential equations to integral equations Solution of integral equations Solution of integral equations Solution of linked equations Solution of linked equations

33 Developing integral equations from differential equations: Simple We now know how to attack integrals with Monte Carlo We now know how to attack integrals with Monte Carlo We desire to be able to “solve” differential equations = estimate functionals (usually integrals or point values) of the function that solves a given equation We desire to be able to “solve” differential equations = estimate functionals (usually integrals or point values) of the function that solves a given equation Traditional solution: Convert them into integral equations and apply the MC integration rules to them Traditional solution: Convert them into integral equations and apply the MC integration rules to them Example: Find the value of f(4), given the differential equation and boundary condition: Example: Find the value of f(4), given the differential equation and boundary condition:

34 Simple integral equations (2) Answer: We can integrate from 0 (the known value) to the desired value to get: Answer: We can integrate from 0 (the known value) to the desired value to get: Now we apply one of the four integration methods to the integral in the equation: Now we apply one of the four integration methods to the integral in the equation:

35 Simple integral equations (2) NOTE: From now on, I will skip the summation and division by N and just write the formula for ONE sample: NOTE: From now on, I will skip the summation and division by N and just write the formula for ONE sample:

36 Simple integral equations (3)  The normal procedure for this method is to: 1. Choose a value of between a and b using a probability distribution  (x) (of YOUR choosing). 2. Score  So, let’s do it.  What PDF should we use?  Lazy man’s PDF: uniform  Optimum PDF: ? (You tell me…)

37 Linked equations When you are faced with linked equation sets, the principles are the same, put you have to be more careful: When you are faced with linked equation sets, the principles are the same, put you have to be more careful: Putting in multiple boundary conditions Putting in multiple boundary conditions Keeping up with multiple sampled variables (each equation will have one) Keeping up with multiple sampled variables (each equation will have one) Most tricky: Realizing and adapting to CHANGING LIMITS on the integrals (after the first) Most tricky: Realizing and adapting to CHANGING LIMITS on the integrals (after the first) MUCH more difficult to optimize the choice of the PDFs used MUCH more difficult to optimize the choice of the PDFs used

38 Linked equation example Example: Find f(2) for the second order differential equation: Example: Find f(2) for the second order differential equation: In order to make it fit the category, we will start be re- writing as the linked set: In order to make it fit the category, we will start be re- writing as the linked set:

39 Linked equation example (2) Applying our tools to the second equation first, we begin by transforming it into an integral equation for the value at x=2: Applying our tools to the second equation first, we begin by transforming it into an integral equation for the value at x=2: Using our MC integration approximation, we get: Using our MC integration approximation, we get: How do we get the ? Answer: We estimate it from the other equation. How do we get the ? Answer: We estimate it from the other equation.

40 Linked equation example (3) Applying our tools to the first equation first, we begin by transforming it into an integral equation for the value at Applying our tools to the first equation first, we begin by transforming it into an integral equation for the value at : The resulting procedure is: The resulting procedure is: 1. Choose a value of using 2. Choose a value of using 3. Score:

41 Linked equation example (4) Now let’s do it. Now let’s do it. What PDF’s to use? What PDF’s to use? Flat Flat Better than flat Better than flat

42 HW

43 HW (2)