Additional Slides on Bayesian Statistics for STA 101 Prof. Jerry Reiter Fall 2008

Can we use this method to learn about means and percentages? To learn about population averages and percentages, we’ve used data (like the DNA test results), but not prior information (like the list of suspects). We show how to combine data and prior information in class.

Combining the prior beliefs and the data using Bayes Rule In Bayes rule problem before break, we combine the prior beliefs and the data using Bayes rule. Pr(p|X=1) represents our posterior beliefs about µ.

Estimation of unknown parameters in statistical models (Bayesian and non-Bayesian) Suppose we posit a probability distribution to model data. How do we estimate its unknown parameters? Example: assume data follow regression model. Where do the estimates of the regression coefficients come from? Classical statistics: maximum likelihood estimation. Bayesian statistics: Bayes rule.

Estimating percentage of Dukies who plan to get advanced degree Suppose we want to estimate the percentage of Duke students who plan to get an advanced degree (MBA, JD, MD, PhD, etc.). Call this percentage p. We sample 20 people at random, and 8 of them say they plan to get an advanced degree. What should be our estimate of p?

Estimating the average IQ of Duke professors Let µ be the population average IQ of Duke profs. Suppose we randomly sample 25 Duke profs and record their IQs. What should be our estimate of µ?

Maximum likelihood estimation: A principled approach to estimation Usually we can use subject-matter knowledge to specify a distribution for the data. But, we don’t know the parameters of that distribution. 1) Number out of 20 who want advanced degree: binomial distribution. 2) Profs’ IQs: normal distribution.

Maximum likelihood estimation We need to estimate the parameters of the distribution. Why do we care? A) So we can make probability statements about future events. B) The parameters themselves may be important.

Maximum likelihood estimation The maximum likelihood estimate of the unknown parameter is the value for which the data were most likely to have occurred. Let’s see how this works in the examples.

Advanced degree example Let Y be the random variable for the number of people out of 20 that plan to get an advanced degree. Y has a binomial distribution with n = 20, and unknown probability p. In the data, Y= 8. If we knew p, the value of the probability distribution function at Y= 8 would be:

MLE for degree example Let’s graph Pr(Y = 8) as a function of the unknown p. Label the function L(p). L(p) is called the likelihood function for p.

Maximum likelihood The maximum likelihood estimate of p is the value of p that maximizes L(p). This is a reasonable estimate because it is the value of p for which the observed data (y= 8 ) had the greatest chance of occurring.

Finding the MLE for degree example To maximize the likelihood function, we need to take the derivative of with respect to p, set it equal to zero, and finally solve for p. You get the sample percentage!

Estimating the average IQ of Duke professors Let µ be the population average IQ of Duke profs. Suppose we randomly sample 25 Duke profs and record their IQs. What should be our estimate of µ?

Model for Professors’ IQs The mathematical function for a normal curve for any prof’s IQ, which we label Y, is: All normal curves have this form, with different means and SDs. Here, we’ll assume the σ = 15. We don’t know µ, which is what we’re after.

Model for all 25 IQs We need the function for all 25 IQs. Assuming each prof’s IQ is independent of other profs’ IQs, we have

Model for all 25 IQs With some algebra and simplifications, the likelihood function is:

Likelihood function and maximum likelihood estimates A graph of the likelihood function looks something like this: The function is maximized when µ is the sample average. So, we use as our estimate of the average Duke prof’s IQ. This sample average is the MLE for µ in any normal curve.

The Bayesian approach to estimation of means Let’s show how to combine data and prior information to address the following motivating question: What is a likely range for the average IQ of Duke professors?

Combining the prior beliefs and the data using Bayes Rule We combine our prior beliefs and the data using Bayes rule. f(µ|data) represents our posterior beliefs about µ.

Formalizing a model for prior information Let’s assign a distribution for µ that reflects our a priori beliefs about its likely range. Label this f(µ). Using the data you supplied in class, the curve describing our beliefs about µ is the normal curve with mean = 128 SD = 15

Mathematical equation for normal curve We can write down the equation for this normal curve.

Model for the data (25 IQs) If we knew µ, the model for the data (the professors’ IQs) is

Estimating the average IQ of Duke professors Let µ be the population average IQ of Duke profs. Suppose we randomly sample 25 Duke profs and record their IQs. What should be our estimate of µ?

Combining the prior beliefs and the data using Bayes Rule We combine the model for the prior beliefs and the model for the data using Bayes rule. f(µ|data) represents our posterior beliefs about µ.

Posterior distribution Using calculus, one can show that f(µ|data) is a normal curve with mean = SD =

Posterior distribution For our data and prior beliefs, the posterior beliefs, f(µ|data), is a normal curve with mean = SD =

Using the posterior distribution to summarize beliefs about µ Because f(µ|data) describes beliefs about µ, we can make probability statements about µ. For example, using a normal curve with mean equal to and SD equal to 2.314, Pr(µ > 130 | data) =.813 A 95% posterior interval for µ stretches from to

Bayesian statistics in general Bayesian methods exist for any population parameter, including percentiles, maxima and minima, ratios, etc. The method is general: 1) specify a mathematical curve that reflects prior beliefs about the population parameter. 2) specify a mathematical curve that describes the distribution of the data, given a value of the population parameter. 3) combine the curves from 1 and 2 mathematically to get posterior beliefs for the parameter, updated for the data.

Differences between frequentist and Bayesian FREQUENTIST  Parameters are not random.  Confidence intervals. BAYESIAN oParameters are random. oPosterior distributions.