Presentation is loading. Please wait.

Presentation is loading. Please wait.

1/21 Finding Optimal Bayesian Network Structures with Constraints Learned from Data 1 City University of New York 2 University of Helsinki Xiannian Fan.

Similar presentations


Presentation on theme: "1/21 Finding Optimal Bayesian Network Structures with Constraints Learned from Data 1 City University of New York 2 University of Helsinki Xiannian Fan."— Presentation transcript:

1 1/21 Finding Optimal Bayesian Network Structures with Constraints Learned from Data 1 City University of New York 2 University of Helsinki Xiannian Fan 1, Brandon Malone 2 and Changhe Yuan 1

2 2/21

3 3/21  Structure Learning of Bayesian Networks Graph Search Formulation Reduce Search Space Using Constraints Experiments Summary Overview

4 4/21 Learning optimal Bayesian networks Very often we have data sets. We can extract knowledge from these data. data structure numerical parameters Structure Learning Graph Search Reduce Search Space Experiments Summary

5 5/21 Score-based learning Structure Learning Graph Search Reduce Search Space Experiments Summary

6 6/21 Graph Search Formulation Formulate the learning task as a shortest path finding problem –The shortest path solution to a graph search problem corresponds to an optimal Bayesian network [Yuan, Malone, Wu, IJCAI-11] Structure Learning Graph Search Reduce Search Space Experiments Summary

7 7/21 Search graph (Order graph) ϕ 123 1,21,32,3 1,2,3 4 1,42,43,4 1,2,41,3,42,3,4 1,2,3,4 Formulation: Search space: Variable subsets Start node: Empty set Goal node: Complete set Edges: Select parents Edge cost: BestScore(X,U) for edge U  U  {X} Task: find the shortest path between start and goal nodes 2 1 3 4 3 2 4 1,4,3,2 [Yuan, Malone, Wu, IJCAI-11] Structure Learning Graph Search Reduce Search Space Experiments Summary

8 8/21 Pruning Based on Bounds ϕ 123 1,32,31,42,4 1,2,41,3,42,3,4 1,2,3,4 Improved search heuristic [Yuan and Malone, UAI-12] Tightening Bounds [Fan, Yuan and Malone, AAA-14] Structure Learning Graph Search Reduce Search Space Experiments Summary

9 9/21 Search Space Size: O(2 n ) ϕ 123 1,21,32,3 1,2,3 4 1,42,4 1,2,41,3,42,3,4 1,2,3,4 3,4 [Yuan, Malone, Wu, IJCAI-11] Structure Learning Graph Search Reduce Search Space Experiments Summary

10 10/21 Motivation for our work Observation: Most Scores are not optimal for any ordering Sparse Parent Graph [Yuan and Malone, UAI-12] –Bounds on Parents Sets for MDL [Tian, UAI-00]) –Properties of Decomposable Score Functions [de Campos and Ji, JMLR-11] Structure Learning Graph Search Reduce Search Space Experiments Summary Most scores can be pruned

11 11/21 Sparse Parent Graph Potential Optimal Parent Set for X 1 Structure Learning Graph Search Reduce Search Space Experiments Summary (a) s 1 (PA 1 ) (b) s 1 (PA 1 ) Propagate the best scores.

12 12/21 Spare Parent Graph Potential Optimal Parent Set for X 1 Structure Learning Graph Search Reduce Search Space Experiments Summary (b) s 1 (PA 1 ) (c) s 1 (PA 1 ) Prune useless local scores.

13 13/21 Motivation for our work Potentially Optimal Parent Sets (POPS) An example: Observation: Not all variables can possibly be ancestors of the others. –E.g., any variables in {X 3,X 4,X 5,X 6 } can not be ancestor of X 1 or X 2 Structure Learning Graph Search Reduce Search Space Experiments Summary

14 14/21 POPS Constraints Parent Child Graph –Formed by Aggregating the POPS Component Graph –Formed by Strongly Connected Components (SCCs) –Give the Ancestor Constraints {1, 2} {3,4,5,6} Structure Learning Graph Search Reduce Search Space Experiments Summary

15 15/21 POPS Constraints Decompose the Problem –Each SCC corresponds to a smaller subproblem –Each subproblem can be solved independently. {1, 2} {3,4,5,6} Structure Learning Graph Search Reduce Search Space Experiments Summary

16 16/21 POPS Constraints Recursive POPS Constraints –Selecting the parents for one of the variables has the effect of removing that variable from the parent relation graph. Structure Learning Graph Search Reduce Search Space Experiments Summary

17 17/21 Top-p POPS Constraints Structure Learning Graph Search Reduce Search Space Experiments Summary

18 18/21 Experiment : POPS and Recursive POPS Constraints Structure Learning Graph Search Reduce Search Space Experiments Summary

19 19/21 Experiment : POPS and Recursive POPS Constraints Structure Learning Graph Search Reduce Search Space Experiments Summary

20 20/21 Experiment : POPS and Recursive POPS Constraints Structure Learning Graph Search Reduce Search Space Experiments Summary

21 21/21 Experiment :Top-p POPS Constraints on Hailfinder Structure Learning Graph Search Reduce Search Space Experiments Summary

22 22/21 Summary: Graph Search Formulation This Work: Reduce Search Space Using POPS Constraints Structure Learning Graph Search Reduce Search Space Experiments Summary


Download ppt "1/21 Finding Optimal Bayesian Network Structures with Constraints Learned from Data 1 City University of New York 2 University of Helsinki Xiannian Fan."

Similar presentations


Ads by Google