1 in data …uncertainty and complexity in models and

2 What do I mean by structure? The key idea is conditional independence: x and z are conditionally independent given y if p(x,z|y) = p(x|y)p(z|y) … implying, for example, that p(x|y,z) = p(x|y) CI turns out to be a remarkably powerful and pervasive idea in probability and statistics

3 How to represent this structure? The idea of graphical modelling: we draw graphs in which nodes represent variables, connected by lines and arrows representing relationships We separate logical (the graph) and quantitative (the assumed distributions) aspects of the model

4 Regression Graphical models Contingency tables Spatial statistics Sufficiency Markov chains Covariance selection Statistical physics Genetics AI

5 Graphical modelling [1] Assuming structure to do probability calculations Inferring structure to make substantive conclusions Structure in model building Inference about latent variables

6 Basic DAG in general: for example:

7 A natural DAG from genetics ABAO OO

8 A natural DAG from genetics ABAO OO AOABAO

9 DNA forensics example (thanks to Julia Mortera) A blood stain is found at a crime scene A body is found somewhere else! There is a suspect DNA profiles on all three - crime scene sample is a ‘mixed trace’: is it a mix of the victim and the suspect?

10 DNA forensics in Hugin Disaggregate problem in terms of paternal and maternal genes of both victim and suspect. Assume Hardy-Weinberg equilibrium We have profiles on 8 STR markers - treated as independent (linkage equilibrium)

11 DNA forensics The data: 2 of 8 markers show more than 2 alleles at crime scene  mixture of 2 or more people

12 DNA forensics in Hugin

13 DNA forensics Population gene frequencies for D7S820 (used as ‘prior’ on ‘founder’ nodes):

14

15 DNA forensics Results (suspect+victim vs. unknown+victim):

16 Graphical modelling [2] Assuming structure to do probability calculations Inferring structure to make substantive conclusions Structure in model building Inference about latent variables

17 Conditional independence graph draw an (undirected) edge between variables  and  if they are not conditionally independent given all other variables   

18 Infant mortality example Data on infant mortality from 2 clinics, by level of ante-natal care (Bishop, Biometrics, 1969) :

19 Infant mortality example Same data broken down also by clinic:

20 Analysis of deviance Resid Resid Df Deviance Df Dev P(>|Chi|) NULL Clinic e-19 Ante Survival e-169 Clinic:Ante e-44 Clinic:Survival e-05 Ante:Survival Clinic:Ante:Survival e

21 Infant mortality example ante clinic survival survival and clinic are dependent and ante and clinic are dependent but survival and ante are CI given clinic

22 Prognostic factors for coronary heart disease strenuous physical work? family history of CHD? strenuous mental work? blood pressure > 140? smoking? ratio of  and  lipoproteins >3? Analysis of a 2 6 contingency table (Edwards & Havranek, Biometrika, 1985)

23 Graphical modelling [3] Assuming structure to do probability calculations Inferring structure to make substantive conclusions Structure in model building Inference about latent variables

24 Modelling with undirected graphs Directed acyclic graphs are a natural representation of the way we usually specify a statistical model - directionally: disease  symptom past  future parameters  data ….. However, sometimes (e.g. spatial models) there is no natural direction

25 Scottish lip cancer data The rates of lip cancer in 56 counties in Scotland have been analysed by Clayton and Kaldor (1987) and Breslow and Clayton (1993) (the analysis here is based on the example in the WinBugs manual)

26 Scottish lip cancer data (2) The data include a covariate measuring the percentage of the population engaged in agriculture, fishing, or forestry, and the "position'' of each county expressed as a list of adjacent counties. the observed and expected cases (expected numbers based on the population and its age and sex distribution in the county),

27 Scottish lip cancer data (3) CountyObsExpxSMR Adjacent casescases(% in counties agric.) ,9,11, , ,24,30,33,45,55

28 Model for lip cancer data (1) Graph observed counts random spatial effects covariate regression coefficient relative risks

29 Model for lip cancer data Data: Link function: Random spatial effects: Priors: (2) Distributions

30 WinBugs for lip cancer data Bugs and WinBugs are systems for estimating the posterior distribution in a Bayesian model by simulation, using MCMC Data analytic techniques can be used to summarise (marginal) posteriors for parameters of interest

31 WinBugs for lip cancer data Dynamic traces for some parameters:

32 WinBugs for lip cancer data Posterior densities for some parameters:

33 Graphical modelling [4] Assuming structure to do probability calculations Inferring structure to make substantive conclusions Structure in model building Inference about latent variables

34 Latent variable problems variable unknownvariable known edges known value set known value set unknown edges unknown

35 Hidden Markov models z0z0 z1z1 z2z2 z3z3 z4z4 y1y1 y2y2 y3y3 y4y4 e.g. Hidden Markov chain observed hidden

36 relative risk parameters Hidden Markov models Richardson & Green (2000) used a hidden Markov random field model for disease mapping observed incidence expected incidence hidden MRF

37 Larynx cancer in females in France SMRs

38 Latent variable problems variable unknownvariable known edges known value set knownvalue set unknown edges unknown

39 Ion channel model choice Hodgson and Green, Proc Roy Soc Lond A, 1999

40 Example hidden continuous time models O2O2 O1O1 C1C1 C2C2 O1O1 O2O2 C1C1 C2C2 C3C3

41 Ion channel model DAG levels & variances model indicator transition rates hidden state data binary signal

42 levels & variances model indicator transition rates hidden state data binary signal O1O1 O2O2 C1C1 C2C2 C3C3 * * * * * * * * * * *

43 Posterior model probabilities O1O1 C1C1 O2O2 O1O1 C1C1 O2O2 O1O1 C1C1 C2C2 O1O1 C1C1 C2C

44 ‘Alarm’ network Learning a Bayesian network, for an ICU ventilator management system, from cases on 37 variables (Spirtes & Meek, 1995)

45 Latent variable problems variable unknown variable known edges known value set knownvalue set unknown edges unknown

46 Wisconsin students college plans sessex pe iq cp 10,318 high school seniors (Sewell & Shah, 1968, and many authors since) 5 categorical variables: sex (2) socioeconomic status (4) IQ (4) parental encouragement (2) college plans (2)

47 sessex pe iq cp 5 categorical variables: sex (2) socioeconomic status (4) IQ (4) parental encouragement (2) college plans (2) (Vastly) most probable graph according to an exact Bayesian analysis by Heckerman (1999)

48 decompos h sessex pe iq cp Heckerman’s most probable graph with one hidden variable