1. Menglong Li Ph.d of Industrial Engineering Dec 1st 2016
Structure estimation for discrete graphical models: Generalized covariance matrices and their inverses
Menglong Li
Ph.d of Industrial Engineering
Dec 1st 2016

2. Outline Recap: Gaussian graphical model
Extend to general graphical model
Model setting
Results
Application
Statistical results

3. Gaussian graphical model
Theorem: For if and only if
encodes the pairwise Markov independencies, and we can recover the graph structure by adding edges with
However, the question of whether a relationship exists between conditional independence and the structure of the inverse covariance matrix in a general graph remains unresolved.

4. How to extend?
Example 1:
Ising model: let the node potentials be for all and the edge potentials be for all ,

5. Continue of Example 1 This matrix is graph structured!
The inverse of covariance matrix is
This matrix is graph structured!

6. How to extend?
Example 2:

7. Continue of Example 2 This matrix is not graph structured!
The inverse of covariance matrix is
This matrix is not graph structured!

8. How to extend?
Example 3:
The inverse of covariance matrix is graph structured!

9. Continue of Example 2
Instead of consider the original graph, we consider the triangulation graph of G as right
we compute the covariance matrix of the
augmented random vector
where the extra term is represented by the
dotted edge shown (the 2-clique )

10. Continue of Example 2
The inverse of this generalized covariance matrix takes the form
If we add all the representation of power set of maximal cliques of triangulated G, compute the covariance of following vector:

11. Continue of Example 2
Empirically, we find that the inverse of the matrix continues to respect aspects of the graph structure: in particular, there are zeros in position , corresponding to the associated functions
and
whenever and do not lie within the same maximal clique

12. Main results
Theorem: Consider an arbitrary discrete graphical model of the form

13. Corollary

14. Statistical results
Suppose we are given n samples drawn from a discrete graphical model, i.i.d. How to recover the graph structure?
Algorithm(Graphical Lasso):
This program is called graphical lasso.

15. Statistical results
Mutual incoherence condition:

16. Reference
Loh, P. L., & Wainwright, M. J. (2013). Structure estimation for discrete graphical models: Generalized covariance matrices and their inverses. The Annals of Statistics, 41(6),