Dependency networks Sushmita Roy BMI/CS Nov 26 th, 2013

Goals for today Introduction to Dependency networks GENIE3: A network inference algorithm for learning a dependency network from gene expression data Comparison of various network inference algorithms

What you should know What are dependency networks? How they differ from Bayesian networks? Learning a dependency network from expression data Evaluation of various network inference methods

Graphical models for representing regulatory networks Bayesian networks Dependency networks Structure Msb 2 Sho1 Ste20 Random variables encode expression levels T ARGET R EGULATORS X1X1 X2X2 Y3Y3 X1X1 X2X2 Y3Y3 Edges correspond to some form of statistical dependencies Y 3 =f(X 1,X 2 ) Function

Dependency network A type of probabilistic graphical model As in Bayesian networks has – A graph component – A probability component Unlike Bayesian network – Can have cyclic dependencies Dependency Networks for Inference, Collaborative Filtering and Data visualization Heckerman, Chickering, Meek, Rounthwaite, Kadie 2000

Notation X i : i th random variable X={X 1,.., X p } : set of p random variables x i k : An assignment of X i in the k th sample x -i k : Set of assignments to all variables other than X i in the k th sample

Dependency networks ??? … XjXj Regulators Function: f j can be of different types. Learning requires estimation of each of the fj functions In all cases it is trying to minimize an error of predicting X j from its neighborhood: fjfj

Different representations of the fj function If X is continuous – f j can be a linear function – f j can be a regression tree – f j can be a random forest An ensemble of trees If X is discrete – f j can be a conditional probability table – f j can be a conditional probability tree

Linear regression Y (output) X (input) Linear regression assumes that output (Y) is a linear function of the input (X) SlopeIntercept

Estimating the regression coefficient Assume we have N training samples We want to minimize the sum of square errors between true and predicted values of the output Y.

An example random forest for predicting gene expression … Ensemble of Regression trees Output 1 Input A selected path for a set of genes Sox6>0.5

Considerations for learning regression trees Assessing the purity of samples under a leaf node – Minimize prediction error – Minimize entropy How to determine when to stop building a tree? – Minimum number of data points at each leaf node – Depth of the tree – Purity of the data points under any leaf node

Algorithm for learning a regression tree Input: Output variable X j, Input variables X j Initialize tree to single node with all samples under node – Estimate m c : the mean of all samples under the node S: sum of squared error Repeat until no more nodes to split – Search over all input variables and split values and compute S for possible splits – Pick the variable and split value that has the highest improvement in error

GENIE3: GEne Network Inference with Ensemble of trees Solves a set of regression problems – One per random variable Models non-linear dependencies Outputs a directed, cyclic graph with a confidence of each edge Focus on generating a ranking over edges rather than a graph structure and parameters Inferring Regulatory Networks from Expression Data Using Tree-Based Methods Van Anh Huynh-Thu, Alexandre Irrthum, Louis Wehenkel, Pierre Geurts, Plos One 2010

GENIE3 algorithm sketch For each gene j, generate input/output pairs – LS j ={(x -j k,x j k ),k=1..N} – Use a feature selection technique on LS j such as tree building to compute w ij for all genes i ≠ j – w ij quantifies the confidence of the edge between X i and X j Generate a global ranking of regulators based on each w ij

GENIE3 algorithm sketch Figure from Huynh-Thu et al.

Feature selection in GENIE3 Random forest to represent the fj Learning the Random forest Generate M=1000 bootstrap samples At each node to be split, search for best split among K randomly selected variables – K was set to p-1 or (p-1) 1/2

Computing the importance weight of each predictor Feature importance is computed at each test node Remember there can be multiple test nodes per regulator For a test node importance is given by the reduction in variance if we make a split on that node Test nodeSet of data samples that reach the test node #S : Size of the set S Var(S): variance of the output variable in set S

Computing the importance of a predictor For a single tree the overall importance is then sum over over all points in the tree where this node is used to split For an ensemble the importance is averaged over all trees.

Computational complexity of GENIE3 Complexity per variable – O(TKNlog N) – T is the number of trees – K is the number of random attributes selected per split – N is the learning sample size

Evaluation of network inference methods Assume we know what the “right” network is One can use Precision-Recall curves to evaluate the predicted network Area under the PR curve (AUPR) curve quantifies performance

AUPR based performance comparison

DREAM: Dialogue for reverse engineeting assessments and methods Community effort to assess regulatory network inference DREAM 5 challenge Previous challenges: 2006, 2007, 2008, 2009, 2010 Marbach et al. 2012, Nature Methods

Where do different methods rank? Marbach et al., 2010 Community Random

Comparing module (LeMoNe) and per-gene (CLR) methods

Summary of network inference methods Probabilistic graphical models provide a natural representation of networks A lot of network inference is done using gene expression data Many algorithms exist, we have seen three – Bayesian networks Sparse candidates Module networks – Dependency networks – GENIE3 Algorithms can be grouped into per-gene and per- module