Bayesian inference review Objective –estimate unknown parameter  based on observations y. Result is given by probability distribution. Bayesian inference –Establish prior of  if any. –Establish likelihood of y conditional on  –Derive posterior distribution Posterior analysis & prediction –Analyze posterior distribution of . –Predict distribution of new y ~ based on posterior distribution . Bayesian updating –with new data, old posterior turned into prior, and repeat process. Prior distribution Likelihood function Observed data Posterior distribution

Bayesian inference for normal distribution Objective –Estimate unknown distribution parameters  of normal distribution based on observations y: mean  & var   (stdev . Cases in this lecture –Observations y that follows normal distribution. –Single observation y –Multiple observations y = {y 1, y 2, …} –Estimate mean  with known variance   –Estimate variance   with known mean 

Estimating mean with known variance from single datum Variance  2 known. Mean  unknown. Single observation y. Likelihood of y: No prior on  Then  ~ constant or just ignore. Posterior density: –With no prior, the distribution of the mean is same as sample distribution. Because of symmetry, no need to re-normalize. Practice –Generate one datum from  =90 with  =10. –Plot the shape of posterior pdf. Superpose original normal distribution.

Predictive distribution Posterior prediction –First term in integrand does not include y, because given theta, y does not matter. –See proof in Bayesian Data Analysis that –Variance is 2  2, which is sum of native variance and variance of mean. Stdev is √2 .

Estimating mean with known variance (conjugate prior) Conjugate prior is normal Posterior density works out to be –Posterior mean is weighted average of prior mean  0 and observed y with weights inversely proportional to the variance –If  0 → ∞ we get back  1 = y,  1 =  Similarly, predictive distribution

Multiple data Independent & identically distributed (iid) observations Posterior density (no prior) Posterior prediction

Practice Observations are 25 data of normal dist where y ̄ 2.9 and stdev  Plot posterior pdf of unknown population mean. What is the probability that the true mean >3 Superpose the original normal distribution, assuming y ̄ is the true mean. Plot cdfs of the two together.

Estimating variance with known mean  (multiple data) In case of no information, a reasonable prior is –Sample distribution (observation) where –Posterior distribution –Equivalent to –Chi square distribution is a standard distribution

Working with Chi square For the posterior pdf we have Obviously, P[  2 ≤ c] is same as P[z ≥ z c =n  c] so for CDF of posterior distribution for given  2 we take one minus the CDF of  2 at zc, in Matlab, 1-chi2cdf(zc,n) By differentiating the CDF, it is easy to show that the posterior pdf is given as Or in Matlab (z/sig2)*chi2pdf(z,n)

Homework Mean with known variance (no prior) generate n=25 samples of normal distribution with sample mean 3 and population stdev 0.2 (you can generate 25 samples and then shift and scale them to get the desired values of mean and standard deviation). 1)write the expression for the posterior distribution of the mean. 2)plot the posterior distribution of the mean using the pdf function and simulation with N=1e6 samples respectively. 3)calculate 2.5%, 97.5% percentiles of the unknown mean from the posterior distribution using the inv function and drawn samples respectively. Variance with known mean (non-informative prior) Use the same samples generated above. 1)write the expression for the posterior distribution of the variance. 2)plot the posterior distribution of the variance using the pdf function and simulation with N=1e6 respectively. 3)calculate 2.5%, 97.5% percentiles of the unknown variance from the posterior distribution using the inv function and drawn samples respectively.