Maximum Likelihood - "Frequentist" inference x 1,x 2,....,x n ~ iid N( ,  2 ) Joint pdf for the whole random sample Maximum likelihood estimates of the model parameters  and  2 are numbers that maximize the joint pdf for the fixed sample which is called the Likelihood function Likelihood function is basically the pdf for the fixed sample

Sampling Distributions x 1,x 2,....,x n ~ iid N( ,  2 ) "Sample Statistics" is a numerical summary of a random sample (e.g. ). As functions of random variables, they are also random variables. "Sampling Distribution" is the probability distribution (statistical model) of "Sample Statistics" - can be derived from the probability distribution of experimental outcomes

"Frequentist" inference Assume that parameters in the model describing the probability of experimental outcome are unknown, but fixed values Given a random sample of experimental outcome (data), we make inference (i.e. make probabilistic statements) about the values of the underlying parameters based on the sampling distributions of parameter estimates and other "sample statistics" Since model parameters are not random variables, these statements are somewhat contrived. For example we don't talk about the p(  >0), but about p(t>t * |  =0). However, for simple situations this works just fine and arguments are mostly philosophical

Bayesian Inference Assumes parameters are random variables - key difference Inference based on the posterior distribution given data Prior Distribution Defines prior knowledge or ignorance about the parameter Posterior Distribution Prior belief modified by data

Bayesian Inference Prior distribution of  Data model given  Posterior distribution of  given data (Bayes theorem) P(  >0|data)

Bayesian Estimation Bayesian point-estimate is the expected value of the parameter under its posterior distribution given data In some cases, the expectation of the posterior distribution could be difficult to assess - easer to find the value for the parameter that maximized the posterior distribution given data - Maximum a Posteriori (MAP) estimate Since the numerator of the posterior distribution in the Bayes theorem is constant in the parameter, this is equivalent to maximizing the product of the likelihood and the prior pdf

Alternative prior for the normal model Degenerate uniform prior for  assuming that any prior value is equally likely - this is clearly unrealistic - we know more than that MAP estimate for  is identical to the maximum likelihood estimate Bayesian point-estimation and maximum likelihood are very closely related

Hierarchical Bayesian Models and Empirical Bayes Inference MOTIVATION x ij ~ ind N(  j,  j 2 ), i=1,...,n is number of replicated observations and j=1,...,T is indexing all genes Each gene has its own mean and variance Usually n is small in comparison to T Want to use information from all genes to estimate the variance of individual gene measurements

Hierarchical Bayesian Models and Empirical Bayes Inference SOLUTION Postulate the "hierarchical" Bayesian model in which individual variances for different genes are assumed to be generated by a single distributions Estimate the parameters of this distribution using the Empirical Bayes approach Estimate individual gene's variances using Bayesian estimation assuming the prior parameters calculated using Empirical Bayes

Hierarchical Bayesian Models and Empirical Bayes Inference Testing the hypothesis  i =0, by calculating the modified t-statistics Limma operates on linear models y j = X  j +  j,  1j,...,  nj ~ N(0,  j 2 ) and the Empirical Bayes estimation is applied to estimate  2 for each gene

Effects of using Empirical Bayes modifications > attributes(FitLMAD) $names [1] "coefficients" "stdev.unscaled" "sigma" "df.residual" "cov.coefficients" [6] "pivot" "method" "design" "genes" "Amean" $class [1] "MArrayLM" attr(,"package") [1] "limma" > attributes(EFitLMAD) $names [1] "coefficients" "stdev.unscaled" "sigma" "df.residual" "cov.coefficients" [6] "pivot" "method" "design" "genes" "Amean" [11] "df.prior" "s2.prior" "var.prior" "proportion" "s2.post" [16] "t" "p.value" "lods" "F" "F.p.value" $class [1] "MArrayLM" attr(,"package") [1] "limma"

Effects of using Empirical Bayes modifications > EFitLMAD$s2.prior [1] > EFitLMAD$df.prior [1]

Effects of using Empirical Bayes modifications > AnovadB$s2.prior [1] > AnovadB$df.prior [1] Empirical Bayes "inflates variances" from the low-variability genes This reduces the proportion of "false positive" resulting from the low variance It biases chance of being differentially expressed towards genes with higher observed differential expressions It has been shown to overall improve the proportion of true positives among the genes pronounced significant "Stein effect" - individually we can not improve over the simple t-test, but by looking at all genes at the same time, turns out that this method works better

Effects of using Empirical Bayes modifications > AnovadB$s2.prior [1] > AnovadB$df.prior [1]

Effects of using Empirical Bayes modifications > AnovadB$s2.prior [1] > AnovadB$df.prior [1]