1. 1/21 Finding Optimal Bayesian Network Structures with Constraints Learned from Data 1 City University of New York 2 University of Helsinki Xiannian Fan 1, Brandon Malone 2 and Changhe Yuan 1

2. 2/21

3. 3/21  Structure Learning of Bayesian Networks Graph Search Formulation Reduce Search Space Using Constraints Experiments Summary Overview

4. 4/21 Learning optimal Bayesian networks Very often we have data sets. We can extract knowledge from these data. data structure numerical parameters Structure Learning Graph Search Reduce Search Space Experiments Summary

5. 5/21 Score-based learning Structure Learning Graph Search Reduce Search Space Experiments Summary

6. 6/21 Graph Search Formulation Formulate the learning task as a shortest path finding problem –The shortest path solution to a graph search problem corresponds to an optimal Bayesian network [Yuan, Malone, Wu, IJCAI-11] Structure Learning Graph Search Reduce Search Space Experiments Summary

7. 7/21 Search graph (Order graph) ϕ 123 1,21,32,3 1,2,3 4 1,42,43,4 1,2,41,3,42,3,4 1,2,3,4 Formulation: Search space: Variable subsets Start node: Empty set Goal node: Complete set Edges: Select parents Edge cost: BestScore(X,U) for edge U  U  {X} Task: find the shortest path between start and goal nodes 2 1 3 4 3 2 4 1,4,3,2 [Yuan, Malone, Wu, IJCAI-11] Structure Learning Graph Search Reduce Search Space Experiments Summary

8. 8/21 Pruning Based on Bounds ϕ 123 1,32,31,42,4 1,2,41,3,42,3,4 1,2,3,4 Improved search heuristic [Yuan and Malone, UAI-12] Tightening Bounds [Fan, Yuan and Malone, AAA-14] Structure Learning Graph Search Reduce Search Space Experiments Summary

9. 9/21 Search Space Size: O(2 n ) ϕ 123 1,21,32,3 1,2,3 4 1,42,4 1,2,41,3,42,3,4 1,2,3,4 3,4 [Yuan, Malone, Wu, IJCAI-11] Structure Learning Graph Search Reduce Search Space Experiments Summary

10. 10/21 Motivation for our work Observation: Most Scores are not optimal for any ordering Sparse Parent Graph [Yuan and Malone, UAI-12] –Bounds on Parents Sets for MDL [Tian, UAI-00]) –Properties of Decomposable Score Functions [de Campos and Ji, JMLR-11] Structure Learning Graph Search Reduce Search Space Experiments Summary Most scores can be pruned

11. 11/21 Sparse Parent Graph Potential Optimal Parent Set for X 1 Structure Learning Graph Search Reduce Search Space Experiments Summary (a) s 1 (PA 1 ) (b) s 1 (PA 1 ) Propagate the best scores.

12. 12/21 Spare Parent Graph Potential Optimal Parent Set for X 1 Structure Learning Graph Search Reduce Search Space Experiments Summary (b) s 1 (PA 1 ) (c) s 1 (PA 1 ) Prune useless local scores.

13. 13/21 Motivation for our work Potentially Optimal Parent Sets (POPS) An example: Observation: Not all variables can possibly be ancestors of the others. –E.g., any variables in {X 3,X 4,X 5,X 6 } can not be ancestor of X 1 or X 2 Structure Learning Graph Search Reduce Search Space Experiments Summary

14. 14/21 POPS Constraints Parent Child Graph –Formed by Aggregating the POPS Component Graph –Formed by Strongly Connected Components (SCCs) –Give the Ancestor Constraints {1, 2} {3,4,5,6} Structure Learning Graph Search Reduce Search Space Experiments Summary

15. 15/21 POPS Constraints Decompose the Problem –Each SCC corresponds to a smaller subproblem –Each subproblem can be solved independently. {1, 2} {3,4,5,6} Structure Learning Graph Search Reduce Search Space Experiments Summary

16. 16/21 POPS Constraints Recursive POPS Constraints –Selecting the parents for one of the variables has the effect of removing that variable from the parent relation graph. Structure Learning Graph Search Reduce Search Space Experiments Summary

17. 17/21 Top-p POPS Constraints Structure Learning Graph Search Reduce Search Space Experiments Summary

18. 18/21 Experiment : POPS and Recursive POPS Constraints Structure Learning Graph Search Reduce Search Space Experiments Summary

19. 19/21 Experiment : POPS and Recursive POPS Constraints Structure Learning Graph Search Reduce Search Space Experiments Summary

20. 20/21 Experiment : POPS and Recursive POPS Constraints Structure Learning Graph Search Reduce Search Space Experiments Summary

21. 21/21 Experiment :Top-p POPS Constraints on Hailfinder Structure Learning Graph Search Reduce Search Space Experiments Summary

22. 22/21 Summary: Graph Search Formulation This Work: Reduce Search Space Using POPS Constraints Structure Learning Graph Search Reduce Search Space Experiments Summary