Sparse Inverse Covariance Estimation with Graphical LASSO J. Friedman, T. Hastie, R. Tibshirani Biostatistics, 2008 Presented by Minhua Chen 1.

Slides:

Advertisements

Similar presentations

Network Inference Umer Zeeshan Ijaz 1. Overview Introduction Application Areas cDNA Microarray EEG/ECoG Network Inference Pair-wise Similarity Measures.

Advertisements

Factorial Mixture of Gaussians and the Marginal Independence Model Ricardo Silva Joint work-in-progress with Zoubin Ghahramani.

Prediction with Regression

Experimental Design, Response Surface Analysis, and Optimization

Chapter 10 Curve Fitting and Regression Analysis

Exploiting Sparse Markov and Covariance Structure in Multiresolution Models Presenter: Zhe Chen ECE / CMR Tennessee Technological University October 22,

Nonlinear Programming

Chapter 2: Lasso for linear models

Proximal Methods for Sparse Hierarchical Dictionary Learning Rodolphe Jenatton, Julien Mairal, Guillaume Obozinski, Francis Bach Presented by Bo Chen,

Second order cone programming approaches for handing missing and uncertain data P. K. Shivaswamy, C. Bhattacharyya and A. J. Smola Discussion led by Qi.

Eric Xing © Eric CMU, Machine Learning Learning Graphical Models Eric Xing Lecture 12, August 14, 2010 Reading:

Unsupervised Feature Selection for Multi-Cluster Data Deng Cai et al, KDD 2010 Presenter: Yunchao Gong Dept. Computer Science, UNC Chapel Hill.

Inverse Regression Methods Prasad Naik 7 th Triennial Choice Symposium Wharton, June 16, 2007.

1 Graphical Models in Data Assimilation Problems Alexander Ihler UC Irvine Collaborators: Sergey Kirshner Andrew Robertson Padhraic Smyth.

Tokyo Research Laboratory © Copyright IBM Corporation 2009 | 2009/04/ | SDM 2009 / Anomaly-Detection Proximity-Based Anomaly Detection using Sparse Structure.

Sparse Word Graphs: A Scalable Algorithm for Capturing Word Correlations in Topic Models Ramesh Nallapati Joint work with John Lafferty, Amr Ahmed, William.

Linear Regression Models Based on Chapter 3 of Hastie, Tibshirani and Friedman Slides by David Madigan.

Temporal Causal Modeling with Graphical Granger Methods

N. Karampetakis, S. Vologiannidis

GAUSSIAN PROCESS REGRESSION FORECASTING OF COMPUTER NETWORK PERFORMANCE CHARACTERISTICS 1 Departments of Computer Science and Mathematics, 2 Department.

Basic Mathematics for Portfolio Management. Statistics Variables x, y, z Constants a, b Observations {x n, y n |n=1,…N} Mean.

Blitz: A Principled Meta-Algorithm for Scaling Sparse Optimization Tyler B. Johnson and Carlos Guestrin University of Washington.

Factor Analysis Ulf H. Olsson Professor of Statistics.

An extension of the compound covariate prediction under the Cox proportional hazard models Emura, Chen & Chen [ 2012, PLoS ONE 7(10) ] Takeshi Emura (NCU)

Today Wrap up of probability Vectors, Matrices. Calculus

Bump Hunting The objective PRIM algorithm Beam search References: Feelders, A.J. (2002). Rule induction by bump hunting. In J. Meij (Ed.), Dealing with.

Thinking Mathematically Algebra: Graphs, Functions and Linear Systems 7.3 Systems of Linear Equations In Two Variables.

Machine Learning CUNY Graduate Center Lecture 3: Linear Regression.

Discriminant Function Analysis Basics Psy524 Andrew Ainsworth.

Computer vision: models, learning and inference Chapter 19 Temporal models.

1 Active learning based survival regression for censored data Bhanukiran Vinzamuri Yan Li Chandan K.

Solving Systems of Equations by Graphing  What is a System of Equations?  Solving Linear Systems – The Graphing Method  Consistent Systems – one point.

Segmental Hidden Markov Models with Random Effects for Waveform Modeling Author: Seyoung Kim & Padhraic Smyth Presentor: Lu Ren.

earthobs.nr.no Temporal Analysis of Forest Cover Using a Hidden Markov Model Arnt-Børre Salberg and Øivind Due Trier Norwegian Computing Center.

BrainStorming 樊艳波 Outline Several papers on icml15 & cvpr15 PALM Information Theory Learning.

EXAMPLE 3 Find the inverse of a 3 × 3 matrix Use a graphing calculator to find the inverse of A. Then use the calculator to verify your result. 2 1 – 2.

Center for Evolutionary Functional Genomics Large-Scale Sparse Logistic Regression Jieping Ye Arizona State University Joint work with Jun Liu and Jianhui.

The Group Lasso for Logistic Regression Lukas Meier, Sara van de Geer and Peter Bühlmann Presenter: Lu Ren ECE Dept., Duke University Sept. 19, 2008.

Bayesian Multivariate Logistic Regression by Sean O’Brien and David Dunson (Biometrics, 2004 ) Presented by Lihan He ECE, Duke University May 16, 2008.

Linear Discriminant Analysis and Its Variations Abu Minhajuddin CSE 8331 Department of Statistical Science Southern Methodist University April 27, 2002.

Lecture 2: Statistical learning primer for biologists

Efficient Gaussian Process Regression for large Data Sets ANJISHNU BANERJEE, DAVID DUNSON, SURYA TOKDAR Biometrika, 2008.

Simulation Study for Longitudinal Data with Nonignorable Missing Data Rong Liu, PhD Candidate Dr. Ramakrishnan, Advisor Department of Biostatistics Virginia.

Mining Brain Region Connectivity for Alzheimer's Disease Study via Sparse Inverse Covariance Estimation Jieping Ye Arizona State University.

Linear Methods for Classification Based on Chapter 4 of Hastie, Tibshirani, and Friedman David Madigan.

Zhilin Zhang, Bhaskar D. Rao University of California, San Diego March 28,

Logistic Regression & Elastic Net

SGPP: Spatial Gaussian Predictive Process Models for Neuroimaging Data Yimei Li Department of Biostatistics St. Jude Children’s Research Hospital Joint.

Robust Optimization and Applications in Machine Learning.

Multi-label Prediction via Sparse Infinite CCA Piyush Rai and Hal Daume III NIPS 2009 Presented by Lingbo Li ECE, Duke University July 16th, 2010 Note:

1 Bios 760R, Lecture 1 Overview  Overview of the course  Classification and Clustering  The “curse of dimensionality”  Reminder of some background.

Estimator Properties and Linear Least Squares

Multi-task learning approaches to modeling context-specific networks

Probability Theory and Parameter Estimation I

Review Problems Matrices

10.8 Systems of Second-Degree Equations and Inequalities

Solving Quadratic Equations by the Complete the Square Method

Menglong Li Ph.d of Industrial Engineering Dec 1st 2016

J. Zhu, A. Ahmed and E.P. Xing Carnegie Mellon University ICML 2009

Propagating Uncertainty In POMDP Value Iteration with Gaussian Process

Learning with information of features

Learning Graphical Models Eric Xing Lecture 12, August 14, 2010

Estimating Networks With Jumps

Exploring Quadratic Expressions

Ch11 Curve Fitting II.

Solving simultaneous linear and quadratic equations

 = N  N matrix multiplication N = 3 matrix N = 3 matrix N = 3 matrix

Drawing Graphs The straight line Example

Learning Incoherent Sparse and Low-Rank Patterns from Multiple Tasks

L5-7 Objective: Students will be able to solve quadratics by using the quadratic formula.

Presentation transcript:

Sparse Inverse Covariance Estimation with Graphical LASSO J. Friedman, T. Hastie, R. Tibshirani Biostatistics, 2008 Presented by Minhua Chen 1

Motivation Mathematical Model Mathematical Tools Graphical LASSO Related papers 2 Outline

Motivation (M. Choi, V. Chandrasekaran and A.S. Willsky, 2009) (O. Banerjee, L. Ghaoui, and A. d’Aspremont, 2008) 3

The optimization problem is concave (M. Yuan and Y. Lin, 2007). Various optimization algorithms have been proposed (M. Yuan and Y. Lin, 2007; O. Banerjee, L. Ghaoui, and A. d’Aspremont, 2008; N. Meinshausen and P. Buhlmann, 2006). The Graphical LASSO algorithm, built on a previous paper (O. Banerjee, L. Ghaoui, and A. d’Aspremont, 2008), is widely used due to its computational efficiency. It transforms the above optimization to LASSO regressions. Mathematical Model 4

Subgradient (J. Tropp, 2006) Mathematical Tools (1) Example 1:Example 2: 5

Mathematical Tools (2) Matrix inversion identity: The above equations reveal the relationship between the inverse covariance matrix and the covariance matrix. 6

Graphical LASSO (1) 7

Graphical LASSO (2) 8

Graphical LASSO (3) 9

Graphical LASSO (4) Ground TruthInferred 10

Related papers: N. Stadler and P. Buhlmann, Missing Values: Sparse Inverse Covariance Estimation and an Extension to Sparse Regression Proposed a MissGLasso algorithm to impute the missing data and infer the inverse covariance matrix simultaneously. O. Banerjee, L. El Ghaoui and A. d’Aspremont, Model Selection Through Sparse Maximum Likelihood Estimation for Multivariate Gaussian or Binary Data Used a constrained quadratic programming algorithm (COVSEL) to solve the same optimization problem as Graphical LASSO. N. Meinshausen and P. Buhlmann, High-Dimensional Graphs and Variable Selection with the Lasso Proposed a neighborhood selection method to approximate the Gaussian Graph. 11