2. 1 Inferring structure to make substantive conclusions: How does it work? Hypothesis testing approaches: Tests on deviances, possibly penalised (AIC/BIC, etc.), MDL, cross-validation... Problem is how to search model space when dimension is large

3. 2 Inferring structure to make substantive conclusions: How does it work? Bayesian approaches: Typically place prior on all graphs, and conjugate prior on parameters (hyper- Markov laws, Dawid & Lauritzen), then use MCMC to update both graphs and parameters to simulate posterior distribution

4. 3 Graph moves Giudici & Green (Biometrika, 1999) develop a full Bayesian methodology for model selection in Gaussian models, assuming decomposability (= graph triangulated = no chordless -cycles) 76 5 23 4 1

5. 4 How many graphs are decomposable? Models using decomposable graphs are ‘dense’ Is decomposability a serious constraint? out of

6. 5 Is decomposability any use? Maximum likelihood estimates can be computed exactly in decomposable models Decomposability is a key to the ‘message passing’ algorithms for probabilistic expert systems (and peeling genetic pedigrees) 1 2 4 3

7. 6 Graph moves We can traverse graph space by adding and deleting single edges Some are OK, but others make graph non-decomposable 76 5 23 4 1

8. 7 Graph moves Frydenberg & Lauritzen (1989) showed that all decomposable graphs are connected by single-edge moves Can we test for maintaining decomposability before committing to making the change? 76 5 23 4 1

9. 8 Cliques A clique is a maximal complete subgraph: here the cliques are {1,2},{2,6,7}, {2,3,6}, and {3,4,5,6} 76 5 23 4 1

10. 9 Deleting edges? Deleting an edge maintains decomposability if and only if it is contained in exactly one clique of the current graph (Frydenberg & Lauritzen) 76 5 23 4 1

11. 10 76 5 23 4 1 12 2672363456 2636 2 a cliqueanother cliquea separator The running intersection property: For any 2 cliques C and D, C  D is a subset of every node between them in the junction tree A graph is decomposable if and only if it can be represented by a junction tree (which is not unique)

12. 11 76 5 23 4 1 12 2672363456 2636 2 a clique another clique a separator A graph is decomposable if and only if it can be represented by a junction tree (which is not unique) The running intersection property: For any 2 cliques C and D, C  D is a subset of every node between them in the junction tree

13. 12 Non-uniqueness of junction tree 76 5 23 4 1 12 2672363456 2636 2

14. 13 76 5 23 4 1 12 2672363456 2636 2 12 2 Non-uniqueness of junction tree

15. 14 Adding edges? (Giudici & Green) Adding an edge (a,b) maintains decomposability if and only if either: 76 5 23 4 1 there exist sets R and T such that a  R and b  T are cliques and R  T is a separator on the path in the junction tree between them a and b are in different connected components, or

16. 15 You can add edge (1,7) since 1  R and 7  T are cliques (with R={2} and T={2,6}) and R  T={2} is a separator on path between them 76 5 23 4 1 12 2672363456 2636 2

17. 16 You cannot add edge (1,4) since the only cliques containing 1 and 4 resp. are {1,2} and {3,4,5,6}, and {2}  {3,5,6} is not a separator on path between them 76 5 23 4 1 12 2672363456 2636 2

18. 17 Adding edges? (Giudici & Green) Adding an edge (a,b) maintains decomposability if and only if either: 76 5 23 4 1 there exist sets R and T such that a  R and b  T are cliques and R  T is a separator on the path in the junction tree between them a and b are in different connected components, or

19. 18 Proof (in connected case)  First suppose that there are no such sets R and T. We have to show that adding edge (a,b) makes graph non- deomposable. Let a  R and b  T be the cliques containing a and b that have shortest connecting path in the junction tree: by assumption, R  T is not a separator (it may be empty): so all separators on the path are proper supersets of R  T. So there is a shortest path in the original graph: a  r  v 1...v k  t  b with k  0, r  R\T, t  T\R and all v’s  R  T. Joining (a,b) will make a chordless (k+4)-cycle, making the graph non-decomposable.

20. 19 You cannot add edge (1,4) since the only cliques containing 1 and 4 resp. are {1,2} and {3,4,5,6}, and {2}  {3,5,6} is not a separator on path between them 76 5 23 4 1 1212 2672363456 263636 2

21. 20 Proof (in connected case)  S bSaS abS Conversely, suppose such sets R and T do exist. We can suppose a  R and b  T are adjacent in the junction tree (otherwise it is quite easy to show that the junction tree can be manipulated until this is true). Let S=R  T, P=R\T and Q=T\R. There are 4 cases according to whether P and Q are empty or not. Both P and Q empty: (it is easy to see that you still have a tree & that running intersection property is maintained)

22. 21 S bSaSP abS bS bSQ aS aSP S bSQaS S bSQaSP aS abSaSP bS bSQabS Neither P nor Q empty: Only Q empty:Only P empty:

23. 22 Once the test is complete, actually committing to adding or deleting the edge is little work 76 5 23 4 1 12 2672363456 2636 2

24. 23 76 5 23 4 1 127 2672363456 2636 27 12 2 It makes only a (relatively) local change to the junction tree Once the test is complete, actually committing to adding or deleting the edge is little work

25. 24 76 5 23 4 1 127 2672363456 2636 27 It makes only a (relatively) local change to the junction tree 6 Once the test is complete, actually committing to adding or deleting the edge is little work

26. 25 76 5 23 4 1 127 267236356 2636 27 It makes only a (relatively) local change to the junction tree 345 35 Once the test is complete, actually committing to adding or deleting the edge is little work

27. 26 Once the test is complete, actually committing to adding or deleting the edge is little work 76 5 23 4 It makes only a (relatively) local change to the junction tree 127 267236356 2636 27 345 35 1 The End