Sum-Product Networks Ph.D. Student ： Li Weizhuo 2015.1.14.

Slides:

Advertisements

Similar presentations

Sum-Product Networks: A New Deep Architecture

Advertisements

Introduction to Graphical Models Brookes Vision Lab Reading Group.

CS188: Computational Models of Human Behavior

Autonomic Scaling of Cloud Computing Resources

Discriminative Training of Markov Logic Networks

Mixture Models and the EM Algorithm

Structured SVM Chen-Tse Tsai and Siddharth Gupta.

Parameter Learning in MN. Outline CRF Learning CRF for 2-d image segmentation IPF parameter sharing revisited.

Dynamic Bayesian Networks (DBNs)

Undirected Probabilistic Graphical Models (Markov Nets) (Slides from Sam Roweis)

3D Human Body Pose Estimation from Monocular Video Moin Nabi Computer Vision Group Institute for Research in Fundamental Sciences (IPM)

Markov Networks.

Learning Markov Network Structure with Decision Trees Daniel Lowd University of Oregon Jesse Davis Katholieke Universiteit Leuven Joint work with:

1 Learning Entity Specific Models Stefan Niculescu Carnegie Mellon University November, 2003.

Graphical Models Lei Tang. Review of Graphical Models Directed Graph (DAG, Bayesian Network, Belief Network) Typically used to represent causal relationship.

ICML 2003 © Sergey Kirshner, UC Irvine Unsupervised Learning with Permuted Data Sergey Kirshner Sridevi Parise Padhraic Smyth School of Information and.

Topic models for corpora and for graphs. Motivation Social graphs seem to have –some aspects of randomness small diameter, giant connected components,..

Gaussians Pieter Abbeel UC Berkeley EECS Many slides adapted from Thrun, Burgard and Fox, Probabilistic Robotics TexPoint fonts used in EMF. Read the TexPoint.

A Unifying Review of Linear Gaussian Models

Sampling Distributions  A statistic is random in value … it changes from sample to sample.  The probability distribution of a statistic is called a sampling.

Computer vision: models, learning and inference Chapter 5 The Normal Distribution.

Machine Learning and Optimization For Traffic and Emergency Resource Management. Milos Hauskrecht Department of Computer Science University of Pittsburgh.

Extensions to message-passing inference S. M. Ali Eslami September 2014.

Cognitive Computer Vision Kingsley Sage and Hilary Buxton Prepared under ECVision Specific Action 8-3

Extracting Places and Activities from GPS Traces Using Hierarchical Conditional Random Fields Yong-Joong Kim Dept. of Computer Science Yonsei.

Step 3: Classification Learn a decision rule (classifier) assigning bag-of-features representations of images to different classes Decision boundary Zebra.

1 A Discriminative Approach to Topic- Based Citation Recommendation Jie Tang and Jing Zhang Presented by Pei Li Knowledge Engineering Group, Dept. of Computer.

1 Naïve Bayes Models for Probability Estimation Daniel Lowd University of Washington (Joint work with Pedro Domingos)

12/07/2008UAI 2008 Cumulative Distribution Networks and the Derivative-Sum-Product Algorithm Jim C. Huang and Brendan J. Frey Probabilistic and Statistical.

Sum-Product Networks CS886 Topics in Natural Language Processing

2 2  Background  Vision in Human Brain  Efficient Coding Theory  Motivation  Natural Pictures  Methodology  Statistical Characteristics  Models.

1 Variable Elimination Graphical Models – Carlos Guestrin Carnegie Mellon University October 11 th, 2006 Readings: K&F: 8.1, 8.2, 8.3,

Computer Vision Group Prof. Daniel Cremers Autonomous Navigation for Flying Robots Lecture 5.2: Recap on Probability Theory Jürgen Sturm Technische Universität.

Overview of the final test for CSC Overview PART A: 7 easy questions –You should answer 5 of them. If you answer more we will select 5 at random.

CPSC 422, Lecture 11Slide 1 Intelligent Systems (AI-2) Computer Science cpsc422, Lecture 11 Oct, 2, 2015.

Community Detection Algorithms: A Comparative Analysis Authors: A. Lancichinetti and S. Fortunato Presented by: Ravi Tiwari.

Probabilistic Graphical Models seminar 15/16 ( ) Haim Kaplan Tel Aviv University.

Math 4030 – 6a Joint Distributions (Discrete)

Towards Total Scene Understanding: Classiﬁcation, Annotation and Segmentation in an Automatic Framework N 工科所錢雅馨 2011/01/16 Li-Jia Li, Richard.

Sum-Product Networks: A New Deep Architecture

Daphne Koller Overview Conditional Probability Queries Probabilistic Graphical Models Inference.

1 Relational Factor Graphs Lin Liao Joint work with Dieter Fox.

Gaussian Process Networks Nir Friedman and Iftach Nachman UAI-2K.

 Dynamic Bayesian Networks Beyond Graphical Models – Carlos Guestrin Carnegie Mellon University December 1 st, 2006 Readings: K&F: 18.1, 18.2,

1 Variable Elimination Graphical Models – Carlos Guestrin Carnegie Mellon University October 15 th, 2008 Readings: K&F: 8.1, 8.2, 8.3,

Statistical Modeling and Learning in Vision --- cortex-like generative models Ying Nian Wu UCLA Department of Statistics JSM, August 2010.

. The EM algorithm Lecture #11 Acknowledgement: Some slides of this lecture are due to Nir Friedman.

Daphne Koller Independencies Bayesian Networks Probabilistic Graphical Models Representation.

Fill-in-The-Blank Using Sum Product Network

Qian Liu CSE spring University of Pennsylvania

Bayes Net Learning: Bayesian Approaches

J. Zhu, A. Ahmed and E.P. Xing Carnegie Mellon University ICML 2009

Graph Based Multi-Modality Learning

Markov Networks.

Linear Regression.

Learning Markov Networks

Markov Networks Independencies Representation Probabilistic Graphical

Complexity Analysis Variable Elimination Inference Probabilistic

Bayesian Networks Independencies Representation Probabilistic

Markov Chain Monte Carlo

المشرف د.يــــاســـــــــر فـــــــؤاد By: ahmed badrealldeen

Simple Sampling Sampling Methods Inference Probabilistic Graphical

10701 Recitation Pengtao Xie

Factorization & Independence

Factorization & Independence

Conditional Random Fields

Markov Networks Independencies Representation Probabilistic Graphical

Junction Trees 3 Undirected Graphical Models

Markov Networks Independencies Representation Probabilistic Graphical

Markov Networks.

Presentation transcript:

Sum-Product Networks Ph.D. Student ： Li Weizhuo

2 Outline  Motivation  Representation  Inference  Learning

3 Motivation  Graphical Models

4 Motivation  Learning Graphical Model

5 Outline  Motivation  Representation  Inference  Learning

6 Representation  What does an SPN mean?  How to use SPNs to represent other networks?  The Context Specific independence(CSI)

7 What Does an SPN mean?

8 A Univariate Distribution is a SPN

9 A Product of SPNs over a Disjoint Variables is an SPN

10 A Weighted Sum of SPNs over the Same variables is an SPN

11 How to use SPNs to represent other networks?  BN  SPN  MN  SPN  Mixture Model  SPN

12 BN → SPN ???

13 BN → SPN

14 BN → SPN

15 BN → SPN ?????

16 MN → SPN

17 Mixture Model → SPN or

18 The Context Specific Independence(CSI)

19 An example in Ontology Matching  SPN  ( Sims  Map| Disjoint 1 )  SPN  ( Sims  Map| Disjoint 0 ) ??????

20 An example in Ontology Matching (Cont) SubClassof Map Disjoint  Context-specific independence SPN  ( Map(Y1,Y2)  Similarities(Y1,Y2))|Disjointwith( Y1,Y2) 1 ) X1 Y1 X2 Y2 Z2

21 Outline  Motivation  Representation  Inference  Learning

22 Inference  All marginals are computable in time linear in size of SPN.  All MAP states are computable in time linear in size of SPN.

23 Compute marginals ????? P(X=0)=?

24 Compute MAP ? ?? ? ? Max

25 Outline  Motivation  Representation  Inference  Learning

26 Learning  Generative weight learning  Discriminative weight learning  Structure Learning

27 Generative weight learning (Poon,H & Domingos, UAI (2011))

28 Random forest Hard EM Generative weight learning (Poon,H & Domingos, UAI (2011))

29 Discriminative weight learning (Gens,R & Domingos, NIPS(2012))

30 Discriminative weight learning (Gens,R & Domingos, NIPS(2012))

31 Discriminative weight learning (Gens,R & Domingos, NIPS(2012))

32 Discriminative weight learning (Gens,R & Domingos, NIPS(2012)) Bottom-Up

33 Discriminative weight learning (Gens,R & Domingos, NIPS(2012))

34 Discriminative weight learning (Gens,R & Domingos, NIPS(2012))

35 Discriminative weight learning (Gens,R & Domingos, NIPS(2012))

36 Discriminative weight learning (Gens,R & Domingos, NIPS(2012))

37 Discriminative weight learning (Gens,R & Domingos, NIPS(2012))

38 Discriminative weight learning (Gens,R & Domingos, NIPS(2012))

39 Discriminative weight learning (Gens,R & Domingos, NIPS(2012))

40 Discriminative weight learning (Gens,R & Domingos, NIPS(2012))

41 Discriminative weight learning (Gens,R & Domingos, NIPS(2012))

42 Discriminative weight learning (Gens,R & Domingos, NIPS(2012))

43 Discriminative weight learning (Gens,R & Domingos, NIPS(2012))

44 Structure Learning (Gens,R & Domingos, ICML(2013)) Mutual information Hard EM

45 Summary Maybe Nothing!

46 Summary

47 References  Most of the materials come from Domingo's slides.  Source code  video _learning/ ?id=192562&r=1

48 Thanks! Q&A