1. Applied Bayesian Inference, KSU, April 29, 2012 §  / §❻ Hierarchical (Multi-Stage) Generalized Linear Models Robert J. Tempelman 1

2. Applied Bayesian Inference, KSU, April 29, 2012 §  / Introduction Some inferential problems require non- classical approaches; e.g. – Heterogeneous variances and covariances across environments. – Different distributional forms (e.g. heavy-tailed or mixtures for residual/random effects). – High dimensional variable selection models Hierarchical Bayesian modeling provides some flexibility for such problems. 2

3. Applied Bayesian Inference, KSU, April 29, 2012 §  / Heterogeneous variance models (Kizilkaya and Tempelman, 2005) Consider a study involving different subclasses (e.g. herds). – Mean responses are different. – But suppose residual variances are different too. Let’s discuss in context of LMM (linear mixed model) 3

4. Applied Bayesian Inference, KSU, April 29, 2012 §  / Recall linear mixed model Given: has a certain “heteroskedastic” specification. determines the nature of heterogeneous residual variances 4

5. Applied Bayesian Inference, KSU, April 29, 2012 §  / Modeling Heterogeneous Variances Suppose – with as a “fixed” intercept residual variance –  k > 0 k th fixed scaling effect. – v l > 0 l th random scaling effect. 5

6. Applied Bayesian Inference, KSU, April 29, 2012 §  / Subjective and Subjective Priors 6

7. Applied Bayesian Inference, KSU, April 29, 2012 §  / Remaining priors “Classical” random effects “Classical” fixed effects “Classical” random effects VC Hyperparameter (Albert, 1988) SAS PROC MCMC doesn’t seem to handle this…prior can’t be written as function of corresponding parameter 7

8. Applied Bayesian Inference, KSU, April 29, 2012 §  / What was the last prior again??? 8 Uniform(0,1) on Different diffuse priors can have different impacts on posterior inferences!...if data info is poor Rosa et al. (2004)

9. Applied Bayesian Inference, KSU, April 29, 2012 §  / Joint Posterior Density LMM: 9

10. Applied Bayesian Inference, KSU, April 29, 2012 §  / Details on FCD All provided by Kizilkaya and Tempelman (2005) – All are recognizeable except for  v : – Use Metropolis-Hastings random walk on using normal as proposal density. For MH, generally a good idea to transform parameters so that parameter space is entire real line…but don’t forget to include Jacobian of transform. 10

11. Applied Bayesian Inference, KSU, April 29, 2012 §  / Small simulation study Two different levels of heterogeneity: –  e = 5,  e = 15 – = 1 Two different average random subclass sizes: – n e = 10 vs. n e = 30 – 20 subclasses (habitats) in total Also modeled fixed effects: – Sex (2 levels) for location and dispersion (  1 =2,  2 =1). Additional set of random effects: – 30 levels (e.g. sires) cross-classified with habitats. 11

12. Applied Bayesian Inference, KSU, April 29, 2012 §  / PROC MIXED code “Fixed” effects models for residual variances – REML estimates of “herd” variances expressed relative to average. proc mixed data=phenotype; class sireid habitatid sexid; model y = sexid; random intercept /subject = habitatid ; random intercept /subject = sireid; repeated / local = exp(sexid habitatid); ods output covparms=covparms; run; 12 but treats v l as a fixed effect. Models

13. Applied Bayesian Inference, KSU, April 29, 2012 §  / MCMC analyses (code available online) Posterior summaries on  e.  e = 15; n e = 10 MeanMedianStd Dev 1st Pctl99th Pctl 58.8420.3699.723.755562.6  e = 5; n e = 10 MeanMedianStd Dev 1st Pctl99th Pctl 4.5313.4163.4282.07322.24  e = 5; n e = 30 MeanMedianStd Dev 1st Pctl99th Pctl 3.6833.3821.3022.0818.006  e = 15; n e = 30 MeanMedianStd Dev 1st Pctl99th Pctl 67.2441.2585.307.918487.5 13

14. Applied Bayesian Inference, KSU, April 29, 2012 §  / MCMC ( ₀ ) and REML () estimates of subclass residual variances vs. truth (v l )  e =15;n e =10  e =5;n e =10  e =5;n e =30  e =15;n e =30 High shrinkage situation Low shrinkage situation 14

15. Applied Bayesian Inference, KSU, April 29, 2012 §  / Heterogeneous variances for ordinal categorical data Suppose we had a situation where residual variances were heterogeneous on the underlying latent scale – i.e., greater frequency of extreme vs. intermediate categories in some subclasses 15

16. Applied Bayesian Inference, KSU, April 29, 2012 §  / Heterogeneous variances for ordinal categorical data? On liability scale: has a certain “heteroskedastic” specification. determines the nature of heterogeneous variances 16

17. Applied Bayesian Inference, KSU, April 29, 2012 §  / Cumulative probit mixed model (CPMM) For CPMM,  maps to Y: 17

18. Applied Bayesian Inference, KSU, April 29, 2012 §  / Modeling Heterogeneous Variances in CPMM Suppose – With as a “fixed” reference residual variance –  k > 0 k th fixed scaling effect. – v l > 0 l th random scaling effect. – All other priors same as with LMM 18

19. Applied Bayesian Inference, KSU, April 29, 2012 §  / Joint Posterior Density in CPMM CPMM: 19

20. Applied Bayesian Inference, KSU, April 29, 2012 §  / Another small simulation study Two different levels of heterogeneity: –  e = 5,  e = 15 Average random subclass size: n e = 30 – 20 subclasses (habitats) in total Also modeled fixed effects: – Sex (2 levels) for location and dispersion. Additional set of random effects: – 30 levels (e.g. sires) cross-classified with habitats. Thresholds:  1 = -1,  1 = 1.5 20

21. Applied Bayesian Inference, KSU, April 29, 2012 §  /  e = 15; n e = 30 MeanMedianStd Dev 1st Pctl99th Pctl 49.4423.2175.315.018404.7 ESS = 391  e = 5; n e = 30 MeanMedianStd Dev 1st Pctl99th Pctl 5.0184.3442.1182.12511.56 ESS = 1422 21  e = 5; n e = 30  e = 15; n e = 30

22. Applied Bayesian Inference, KSU, April 29, 2012 §  / Posterior means of subclass residual variances vs. truth (v l )  e =5;n e =30  e =15;n e =30 22 No PROC GLIMMIX counterpart Another alternative: Heterogeneous thresholds!!! (Varona and Hernandez, 2006)

23. Applied Bayesian Inference, KSU, April 29, 2012 §  / Additional extensions PhD work by Fernando Cardoso – Heterogeneous residual variances as functions of multiple fixed effects and multiple random effects. – Heterogeneous t-error (Cardoso et al., 2007). – Helps separates outliers from high variance subclasses from effects of outliers. – Other candidates for distribution of w j lead to alternative heavy-tailed specifications (Rosa et al., 2004) 23 t-error is outlier robust

24. Applied Bayesian Inference, KSU, April 29, 2012 §  / Posterior densities of breed-group heritabilities in multibreed Brazilian cattle (Fernando Cardoso) Some of most variable herds were exclusively Herefords Based on homogeneous residual variance (Cardoso and Tempelman, 2004) Based on heterogeneous residual variances (Fixed: breed additive&dominance,sex; Random: CG (Cardoso et al., 2005) Estimated CV of CG-specific  2 e → 0.72±0.06 F 1  2 e = 0.70±0.16 purebred  2 e

25. Applied Bayesian Inference, KSU, April 29, 2012 §  / Heterogeneous G-side scale parameters Could be accommodated in a similar manner. In fact, the borrowing of information across subclasses in estimating subclass-specific random effects variances is even more critical. – Low information per subclass? REML estimates will converge to zero. 25

26. Applied Bayesian Inference, KSU, April 29, 2012 §  / Heterogeneous bivariate G-side and R-side inferences! Bello et al. (2010, 2012) Investigated herd-level and cow-level relationship between 305-day milk production and calving interval (CI) as a function of various factors CG (herd-year) effects Residual (cow) effects

27. Applied Bayesian Inference, KSU, April 29, 2012 §  / Herd-Specific and Cow- Specific (Co)variances Herd k Cow j Let and

28. Applied Bayesian Inference, KSU, April 29, 2012 §  / Rewrite this Herd k Cow j Model each of these different colored terms as functions of fixed and random effects (in addition to the classical  and u)!

29. Applied Bayesian Inference, KSU, April 29, 2012 §  / bST effect on Herd-Level Association between Milk Yield and Calving Interval 0% <50% ≥50% % herd on bST supplementation days per 100 kg milk yield 0.01 a 0.07 a -1.37 b a,b P < 0.0001 bST: Bovine somatotropin

30. Applied Bayesian Inference, KSU, April 29, 2012 §  / Number of times milking/day on Cow-level Association between Milk Yield and Calving Interval 2X 3 + X Daily Milking Frequency days per 100 kg milk yield 0.57 a 0.45 b a,b P < 0.0001 Overall Antagonism 0.51±0.01 day longer CI per 100 kg increase in cumulative 305-d milk yield

31. Applied Bayesian Inference, KSU, April 29, 2012 §  / Variability between Herds for (Random effects) DIC M0 – DIC M1 = 243 Expected range between extreme herd-years ± 2 = 0.7 d / 100 kg Ott and Longnecker, 2001 0.00.20.40.6 0.81.0 Increase in # of days of CI / 100 kg herd milk yield 0.16 0.7 d/100kg 0.86

32. Applied Bayesian Inference, KSU, April 29, 2012 §  / Whole Genome Selection (WGS) Model: i = 1,2,…,n. (e.g. age, parity) Genotypes SNP allelic substitution effects Polygenic Effects Residual effects Phenotype LD (linkage disequilibrium) Phenotypes Animal Genotypes m >>>>>n

33. Applied Bayesian Inference, KSU, April 29, 2012 §  / Typical WGS specifications Random effects spec. on g (Meuwissen et al. 2001) – BLUP: – BayesA/B: BayesA = BayesB with  = 0. – “Random effects/Bayes” modeling allows m >> n Borrowing of information across genes.

34. Applied Bayesian Inference, KSU, April 29, 2012 §  / First-order antedependence-specifications (Yang and Tempelman, 2012) Instead of independence, specify first order antedependence: SNP MarkerGenetic Effect SNP 1: g 1 =  1, SNP 2: g 2 = t 21 g 1 +  2, SNP 3: g 3 = t 32 g 2 +  3, SNP m: g m = t m,m-1 g m-1 +  m. Ante-BayesB Ante-BayesA = Ante-BayesB with  = 0 Correlation Random effects modeling: facilitates borrowing of information across SNP intervals SNP 1 SNP 2 SNP 3 SNP 4

35. Applied Bayesian Inference, KSU, April 29, 2012 §  / Results from a simulation study Advantage of Ante- BayesA/B over conventional BayesA/B increases with increasing marker density (LD = linkage disequilbrium) Accuracy of Genomic EBV vs. LD level (r 2 ) P<.001 Bayes A/B vs. AnteBayesA/B

36. Applied Bayesian Inference, KSU, April 29, 2012 §  / Other examples of multi-stage hierarchical modeling? Spatial variability in agronomy using t-error (Besag and Higdon, 1999) Ecology (Cressie et al., 2009). Conceptually, one could model heterogeneous and spatially correlated overdispersion parameters in Poisson/binomial GLMM as well! 36

37. Applied Bayesian Inference, KSU, April 29, 2012 §  / What I haven’t covered in this workshop Model choice criteria – Bayes factors (generally, too challenging to compute) – DIC (Deviance information criteria) Bayesian model averaging – Advantage over conditioning on one model (e.g. for multiple regression involving many covariates) Posterior predictive checks. – Great for diagnostics Residual diagnostics based on latent residuals for GLMM (Johnson and Albert, 1999). 37

38. Applied Bayesian Inference, KSU, April 29, 2012 §  / Some closing comments/opinions Merit of Bayesian inference – Marginal for LMM with classical assumptions. GLS with REML seems to work fine. – Of greater benefit for GLMM Especially binary data with complex error structures – Greatest benefit for multi-stage hierarchical models. Larger datasets nevertheless required than with more classical (homogeneous assumptions). 38

39. Applied Bayesian Inference, KSU, April 29, 2012 §  / Implications Increased programming capabilities/skills are needed. – Cloud/cluster computing wouldn’t hurt. Don’t go in blind with canned Bayesian software. – Watch the diagnostics (e.g. trace plots) like a hawk! Don’t go on autopilot. – WinBugs/PROC MCMC works nicely for the simpler stuff. – Highly hierarchical models require statistical/algorithmic insights…do recognize limitations in parameter identifiability (Cressie et al., 2009) 39

40. Applied Bayesian Inference, KSU, April 29, 2012 §  / National Needs PhD Fellowships Michigan State University Focus: Integrated training in quantitative, statistical and molecular genetics, and breeding of food animals Features: Research in animal genetics/genomics with collaborative faculty team Industry internship experience Public policy internship in Washington, DC Statistical consulting center experience Teaching or Extension/outreach learning opportunities Optional affiliation with inter-departmental programs in Quantitative Biology, Genetics, others Faculty Team: C. Ernst, J. Steibel, R. Tempelman, R. Bates, H. Cheng, T. Brown, B. Alston-Mills Eligibility is open to citizens and nationals of the US. Women and underrepresented groups are encouraged to apply.

41. Applied Bayesian Inference, KSU, April 29, 2012 §  / Thank You!!! Any more questions??? 41 http://actuary-info.blogspot.com/2011/05/homo-actuarius- bayesianes.html