Bayesian Model Selection in Factorial Designs Seminal work is by Box and Meyer Seminal work is by Box and Meyer Intuitive formulation and analytical approach, but the devil is in the details! Intuitive formulation and analytical approach, but the devil is in the details! Look at simplifying assumptions as we step through Box and Meyer’s approach Look at simplifying assumptions as we step through Box and Meyer’s approach One of the hottest areas in statistics for several years One of the hottest areas in statistics for several years

Bayesian Model Selection in Factorial Designs There are 2 k-p -1 possible (fractional) factorial models, denoted as a set {M l }. There are 2 k-p -1 possible (fractional) factorial models, denoted as a set {M l }. To simplify later calculations, we usually assume that the only active effects are main effects, two-way effects or three-way effects To simplify later calculations, we usually assume that the only active effects are main effects, two-way effects or three-way effects –This assumption is already in place for low- resolution fractional factorials

Bayesian Model Selection in Factorial Designs Each M l denotes a set of active effects (both main effects and interactions) in a hierarchical model. Each M l denotes a set of active effects (both main effects and interactions) in a hierarchical model. We will use X ik =1 for the high level of effect k and X ik =-1 for the low level of effect k. We will use X ik =1 for the high level of effect k and X ik =-1 for the low level of effect k.

Bayesian Model Selection in Factorial Designs We will assume that the response variables have a linear model with normal errors given model M We will assume that the response variables have a linear model with normal errors given model M X i and  are model-specific, but we will use a saturated model in what follows X i and  are model-specific, but we will use a saturated model in what follows

Bayesian Model Selection in Factorial Designs The likelihood for the data given the parameters has the following form The likelihood for the data given the parameters has the following form

Bayesian Paradigm Unlike in classical inference, we assume the parameters, , are random variables that have a prior distribution, f  (  ), rather than being fixed unknown constants. Unlike in classical inference, we assume the parameters, , are random variables that have a prior distribution, f  (  ), rather than being fixed unknown constants. In classical inference, we estimate  by maximizing the likelihood L(  |y) In classical inference, we estimate  by maximizing the likelihood L(  |y)

Bayesian Paradigm Estimation using the Bayesian approach relies on updating our prior distribution for  after collecting our data y. The posterior density, by an application of Bayes rule, is proportional to the familiar data density and the prior density: Estimation using the Bayesian approach relies on updating our prior distribution for  after collecting our data y. The posterior density, by an application of Bayes rule, is proportional to the familiar data density and the prior density:

Bayesian Paradigm The Bayes estimate of  minimizes Bayes risk—the expected value (with respect to the prior) of loss function L(  ). The Bayes estimate of  minimizes Bayes risk—the expected value (with respect to the prior) of loss function L(  ). Under squared error loss, the Bayes estimate is the mean of the posterior distribution: Under squared error loss, the Bayes estimate is the mean of the posterior distribution:

Bayesian Model Selection in Factorial Designs The Bayesian prior for models is quite straightforward. If r effects are in the model, then they are active with prior probability  The Bayesian prior for models is quite straightforward. If r effects are in the model, then they are active with prior probability 

Bayesian Model Selection in Factorial Designs Since we’re using a Bayesian approach, we need priors for  and  as well Since we’re using a Bayesian approach, we need priors for  and  as well

Bayesian Model Selection in Factorial Designs For non-orthogonal designs, it’s common to use Zellner’s g-prior for  : For non-orthogonal designs, it’s common to use Zellner’s g-prior for  : Note that we did not assign priors to  or  Note that we did not assign priors to  or 

Bayesian Model Selection in Factorial Designs We can combine f( , ,  ) and f(Y| , ,  ) to obtain the full likelihood L( , , ,Y) We can combine f( , ,  ) and f(Y| , ,  ) to obtain the full likelihood L( , , ,Y)

Bayesian Model Selection in Factorial Designs

Our goal is to derive the posterior distribution of M given Y, which first requires integrating out  and . Our goal is to derive the posterior distribution of M given Y, which first requires integrating out  and .

Bayesian Model Selection in Factorial Designs The first term is a penalty for model complexity (smaller is better) The first term is a penalty for model complexity (smaller is better) The second term is a measure of model fit (smaller is better) The second term is a measure of model fit (smaller is better)

Bayesian Model Selection in Factorial Designs  and  are still present. We will fix  ; the method is robust to the choice of   and  are still present. We will fix  ; the method is robust to the choice of   is selected to minimize the probability of no active factors  is selected to minimize the probability of no active factors

Bayesian Model Selection in Factorial Designs With L(M|Y) in hand, we can actually evaluate the P(M i |Y) for all M i for any prior choice of , provided the number of M i is not burdensome With L(M|Y) in hand, we can actually evaluate the P(M i |Y) for all M i for any prior choice of , provided the number of M i is not burdensome This is in part why we assume eligible M i only include lower order effects. This is in part why we assume eligible M i only include lower order effects.

Bayesian Model Selection in Factorial Designs Greedy search or MCMC algorithms are used to select models when they cannot be itemized Greedy search or MCMC algorithms are used to select models when they cannot be itemized Selection criteria include Bayes Factor, Schwarz criterion, Bayesian Information Criterion Selection criteria include Bayes Factor, Schwarz criterion, Bayesian Information Criterion Refer to R package BMA and bic.glm for fitting more general models. Refer to R package BMA and bic.glm for fitting more general models.

Bayesian Model Selection in Factorial Designs For each effect, we sum the probabilities for all M i that contain that effect and obtain a marginal posterior probability for that effect. For each effect, we sum the probabilities for all M i that contain that effect and obtain a marginal posterior probability for that effect. These marginal probabilities are relatively robust to the choice of . These marginal probabilities are relatively robust to the choice of .

Case Study Violin data* (2 4 factorial design with n=11 replications) Violin data* (2 4 factorial design with n=11 replications) Response: Decibels Response: Decibels Factors Factors –A: Pressure (Low/High) –B: Placement (Near/Far) –C: Angle (Low/High) –D: Speed (Low/High) *Carla Padgett, STAT 706 taught by Don Edwards

Case Study Fractional Factorial Design: A, B, and D significant A, B, and D significant AB marginal AB marginal *Carla Padgett, STAT 706 taught by Don Edwards Bayesian Model Selection: A, B, D, AB, AD, BD significant A, B, D, AB, AD, BD significant All others negligible All others negligible