1. Monte Carlo Approximations – Introduction
Suppose X1, X2,… is a sequence of independent and identically distributed random variables with unknown mean µ.
Let
The laws of large numbers indicate that for large n, Mn ≈ µ.
Therefore, it is possible to use Mn as an estimator or approximation of µ.
Estimators like Mn can also be used to estimate purely mathematical quantities that are too difficult to compute directly.
Such estimators are called Monte Carlo approximations.
STA347 - week 12 1

2. Example Suppose we wish to evaluate the integral
This integral cannot easily be solved exactly, but it can be computed
approximately using a Monte Carlo approximation.
We first note that, I = E(X 2 cos(X 2)) where X ~ Exponential(25).
Hence, for large n the integral I is approximately equal to ,
where , with X1, X2,…i.i.d
Exponential(25).
Further, in the example on slide 10, we established a method to
simulate X ~ Exponential(25). 2
STA347 - week 12

3. Putting things together we obtain the following algorithm for
approximating the integral I.
Step 1: Select a large positive integer n.
Step 2: Obtain Ui ~ Uniform[0, 1], independently for i = 1, …, n.
Step 3: Set , for i = 1, …, n.
Step 4: Set , for i = 1, …, n.
Step 5: Estimate I by
For large n this algorithm will provide a good estimate of the integral I.
STA347 - week 12 3

4. Example Using R
The following is the R code for approximating the integral I in the example above.
> U = runif(100000)
> X = -(1/25)*log((1-U), base = exp(1))
> T = X^2*cos(X^2)
> I = mean(T)
> I
[1]
STA347 - week 12 4

5. Assessing Error of MC Approximations
Any time we approximate or estimate a quantity, we must also
indicate how much error there is in the estimate.
However, we cannot say what the error is exactly since we are
approximating an unknown quantity.
Nevertheless, the central limit theorem provide a natural approach to assessing this error, using three times the standard error of the estimate.
Thus, we can approximate an unknown quantity such as the integral I in the example above by quoting Mn and the interval
STA347 - week 12 5

6. Assessing Error Using R
The following is the R code for assessing the error in the
approximation of the integral I.
> llimit = I - 3*sd(T)/sqrt(100000)
> llimit
[1]
> ulimit = I + 3*sd(T)/sqrt(100000)
> ulimit
[1]
Conclusion: the value of I is approximately and the true value is almost certainly in the interval ( , ).
STA347 - week 12 6

7. Conditional Probability on a joint discrete distribution
Given the joint pmf of X and Y, we want to find
and
These are the base for defining conditional distributions…
STA347 - week 12

8. Definition
For X, Y discrete random variables with joint pmf pX,Y(x,y) and marginal mass
function pX(x) and pY(y). If x is a number such that pX(x) > 0, then the
conditional pmf of Y given X = x is
Is this a valid pmf?
Similarly, the conditional pmf of X given Y = y is
Note, from the above conditional pmf we get
Summing both sides over all possible values of Y we get
This is an extremely useful application of the law of total probability.
Note: If X, Y are independent random variables then PX|Y(x|y) = PX(x).
STA347 - week 12

9. Example
Suppose we roll a fair die; whatever number comes up we toss a coin that many times. What is the distribution of the number of heads?
Let X = number of heads, Y = number on die.
We know that
Want to find pX(x).
The conditional probability function of X given Y = y is given by
for x = 0, 1, …, y.
By the Law of Total Probability we have
Possible values of x: 0,1,2,…,6.
STA347 - week 12

10. Conditional densities
If X, Y jointly distributed continuous random variables, the conditional density function of Y | X is defined to be
if fX(x) > 0 and 0 otherwise.
If X, Y are independent then
Also,
Integrating both sides over x we get
This is a useful application of the law of total probability for the continuous case.
STA347 - week 12

11. Example Consider the joint density
Find the conditional density of X given Y and the conditional density of Y given X.
STA347 - week 12

12. Conditional Expectation
For X, Y discrete random variables, the conditional expectation of Y given
X = x is
and the conditional variance of Y given X = x is
where these are defined only if the sums converges absolutely.
In general,
STA347 - week 12

13. For X, Y continuous random variables, the conditional expectation of
Y given X = x is
and the conditional variance of Y given X = x is
In general,
STA347 - week 12

14. Example
Suppose X, Y are continuous random variables with joint density function
Find E(X | Y = 2).
STA347 - week 12

15. More on Conditional Expectation
Assume that E(Y | X = x) exists for every x in the range of X. Then,
E(Y | X ) is a random variable. The expectation of this random variable is
E [E(Y | X )]
Theorem
E [E(Y | X )] = E(Y)
This is called the “Law of Total Expectation”.
Proof:
STA347 - week 12

16. Example
Suppose we roll a fair die; whatever number comes up we toss a coin that many times. What is the expected number of heads?
STA347 - week 12

17. Theorem For random variables X, Y V(Y) = V [E(Y|X)] + E[V(Y|X)] Proof:
STA347 - week 12

18. Example Let X ~ Geometric(p).
Given X = x, let Y have conditionally the Binomial(x, p) distribution.
Scenario: doing Bernoulli trails with success probability p until 1st success so X : number of trails. Then do x more trails and count the number of success which is Y.
Find, E(Y), V(Y).
STA347 - week 12