Presentation is loading. Please wait.

Presentation is loading. Please wait.

Learning to Rank Typed Graph Walks: Local and Global Approaches

Published byNiko Hiltunen Modified over 5 years ago

Similar presentations

Presentation on theme: "Learning to Rank Typed Graph Walks: Local and Global Approaches"— Presentation transcript:

1 Learning to Rank Typed Graph Walks: Local and Global Approaches
Einat Minkov and William W. Cohen Language Technologies Institute and Machine Learning Department School of Computer Science Carnegie Mellon University

2 Did I forget to invite anyone for this meeting?

3 Did I forget to invite anyone for this meeting?
What is Jason’s personal address ?

4 Did I forget to invite anyone for this meeting?
What is Jason’s personal address ? Who is “Mike” who is mentioned in this ?

5 CALO William graph proposal CMU 6/17/07 6/18/07 einat@cs.cmu.edu
Has Subject Term Sent To William graph proposal CMU 6/17/07 6/18/07

6 Q: “what are Jason’s email aliases?”
Sent To Has term inverse “Jason” einat Sent-to Msg 18 Msg5 Msg 2 Jason Ernst Sent to Sent from Alias cs.cmu.edu andrew.cmu.edu Similar to

7 Search via lazy random graph walks
An extended similarity measure via graph walks:

8 Search via lazy random graph walks
An extended similarity measure via graph walks: Propagate “similarity” from start nodes through edges in the graph – accumulating evidence of similarity over multiple connecting paths.

9 Search via lazy random graph walks
An extended similarity measure via graph walks: Propagate “similarity” from start nodes through edges in the graph – accumulating evidence of similarity over multiple connecting paths. Fixed probability of halting the walk at every step – i.e., shorter connecting paths have greater importance (exponential decay)

10 Search via lazy random graph walks
An extended similarity measure via graph walks: Propagate “similarity” from start nodes through edges in the graph – accumulating evidence of similarity over multiple connecting paths. Fixed probability of halting the walk at every step – i.e., shorter connecting paths have greater importance (exponential decay) Finite graph walk, applied through sparse matrix multiplication (estimated via sampling for large graphs)

11 Search via lazy random graph walks
An extended similarity measure via graph walks: Propagate “similarity” from start nodes through edges in the graph – accumulating evidence of similarity over multiple connecting paths. Fixed probability of halting the walk at every step – i.e., shorter connecting paths have greater importance (exponential decay) Finite graph walk, applied through sparse matrix multiplication (estimated via sampling for large graphs) The result is a list of nodes, sorted by “similarity” to an input node distribution (final node probabilities).

12 The graph Graph nodes are typed.
Graph edges - directed and typed (adhering to the graph schema) Multiple relations may hold between two given nodes. Every edge type is assigned a fixed weight.

13 Returns a list of nodes (of type ) ranked by the graph walk probs.
A query language: Q: { , } Graph walks Returns a list of nodes (of type ) ranked by the graph walk probs. The graph Nodes Node type Edge label Edge weight graph walk controlled by edge weights Θ , walk length K and stay probability γ The probability of reaching y from x in one step: the sum of edge weights from x to y, out of the total outgoing weight from x. The transition matrix assumes a stay probability at the current node at every time step.

14 Tasks Person name disambiguation [ term “andy” file msgId ] “person”
Threading What are the adjacent messages in this thread? A proxi for finding generally related messages. [ file msgId ] “ -file” Alias finding [ term Jason ] What are the -addresses of Jason ?... “ -address”

15 Learning to Rank Typed Graph Walks

16 Learning settings GRAPH WALK Task T (query class) Query a Query b
… Query q + Rel. answers a + Rel. answers b + Rel. answers q GRAPH WALK node rank 1 node rank 2 node rank 3 node rank 4 … node rank 10 node rank 11 node rank 12 node rank 50 node rank 1 node rank 2 node rank 3 node rank 4 … node rank 10 node rank 11 node rank 12 node rank 50 node rank 1 node rank 2 node rank 3 node rank 4 … node rank 10 node rank 11 node rank 12 node rank 50

17 Learning approaches Edge weight tuning: Graph walk Weight update
Theta*

18 Learning approaches task Edge weight tuning: Graph walk Weight update
Theta* Graph walk task

19 Learning approaches task Edge weight tuning: Graph walk Weight update
Theta* Graph walk task Node re-ordering: Graph walk Feature generation Update re-ranker Re-ranking function

20 Learning approaches task task Edge weight tuning: Graph walk
Weight update Theta* Graph walk task Node re-ordering: Graph walk Feature generation Update re-ranker Re-ranking function Graph walk Feature generation Score by re-ranker task

21 Learning approaches Graph parameters’ tuning Gradient descent (Chang et-al, 2000) Can be adapted from extended PageRank settings to finite graph walks. Exhaustive local search over edge type (Nie et-al, 05) Hill climbing error backpropagation (Dilligenti et-al, IJCAI-05) Strong assumption of first-order Markov dependencies Gradient descent approximation for partial order preferences (Agarwal et-al, KDD-06) Node re-ordering A discriminative learner, using graph-paths describing features. Re-ranking (Minkov, Cohen and NG, SIGIR-06) Loses some quantitative data in feature decoding. However, can represent edge sequences.

22 Error Backpropagation
following Dilligenti et-al, 2005 Cost function: Weight updates: Where,

23 Re-ranking follows closely on (Collins and Koo, Computational Linguistics, 2005) Scoring function: Adapt weights to minimize (boosted version): , where

24 Path describing Features
X1 ‘Edge unigram’ was edge type l used in reaching x from Vq? X2 X3 ‘Edge (n-)bigram’ were edge types l1 and l2 traversed (in that order) in reaching x from Vq? X4 X5 K=0 K=1 K=2 ‘Top edge (n-)bigram’ same, where only the top k contributing paths are considered. x2  x1  x3 x4  x1  x3 x4  x2  x3 x2  x3 Paths [x3, k=2]: ‘Source count’ indicates the number of different source nodes in the set of connecting paths.

25 Learning to Rank Typed Graph Walks: Local vs. Global approaches

26 Experiments Methods: Tasks & Corpora : Gradient descent: Θ0  ΘG
Reranking: R(Θ0) Combined: R(ΘG) Tasks & Corpora :

27 The results (MAP) Name disambiguation * * * * * Threading * * * * * *
+ + * * Threading * * * * * * * + + + Alias finding MAP

28 Name disambiguation Threading Alias finding

29 Our Findings Re-ranking often preferable due to ‘global’ features:
Models relation sequences. e.g., threading: sent-from  sent-to-inv Re-ranking rewards nodes for which the set of connecting paths is diverse. source-count feature informative for complex queries The approaches are complementary Future work: Re-ranking: large feature space. Re-ranking requires decoding at run-time. Domain specific features

30 Related papers Einat Minkov, William W. Cohen, Andrew Y. Ng Contextual Search and Name Disambiguation in using Graphs SIGIR 2006 Einat Minkov, William W. Cohen An and Meeting Assistant using Graph Walks CEAS 2006 Alekh Agarwal, Soumen Chakrabarti Learning Random Walks to Rank Nodes in Graphs ICML 2007 Hanghang Tong, Yehuda Koren, and Christos Faloutsos Fast Direction-Aware Proximity for Graph Mining KDD 2007

31 Thanks! Questions?

Download ppt "Learning to Rank Typed Graph Walks: Local and Global Approaches"

Similar presentations

Size-estimation framework with applications to transitive closure and reachability Presented by Maxim Kalaev Edith Cohen AT&T Bell Labs 1996.

Size-estimation framework with applications to transitive closure and reachability Presented by Maxim Kalaev Edith Cohen AT&T Bell Labs 1996.
School of Computer Science Carnegie Mellon University Duke University DeltaCon: A Principled Massive- Graph Similarity Function Danai Koutra Joshua T.

School of Computer Science Carnegie Mellon University Duke University DeltaCon: A Principled Massive- Graph Similarity Function Danai Koutra Joshua T.
University of Texas at Austin Machine Learning Group Department of Computer Sciences University of Texas at Austin Discriminative Structure and Parameter.

University of Texas at Austin Machine Learning Group Department of Computer Sciences University of Texas at Austin Discriminative Structure and Parameter.
Active Learning for Streaming Networked Data Zhilin Yang, Jie Tang, Yutao Zhang Computer Science Department, Tsinghua University.

Active Learning for Streaming Networked Data Zhilin Yang, Jie Tang, Yutao Zhang Computer Science Department, Tsinghua University.
Social networks, in the form of bibliographies and citations, have long been an integral part of the scientific process. We examine how to leverage the.

Social networks, in the form of bibliographies and citations, have long been an integral part of the scientific process. We examine how to leverage the.
Hidden Markov Models Theory By Johan Walters (SR 2003)

Hidden Markov Models Theory By Johan Walters (SR 2003)
DATA MINING LECTURE 12 Link Analysis Ranking Random walks.

DATA MINING LECTURE 12 Link Analysis Ranking Random walks.
Review of : Yoav Freund, and Robert E

Review of : Yoav Freund, and Robert E
Fast Query Execution for Retrieval Models based on Path Constrained Random Walks Ni Lao, William W. Cohen Carnegie Mellon University 2010.7.27.

Fast Query Execution for Retrieval Models based on Path Constrained Random Walks Ni Lao, William W. Cohen Carnegie Mellon University
Fast Direction-Aware Proximity for Graph Mining KDD 2007, San Jose Hanghang Tong, Yehuda Koren, Christos Faloutsos.

Fast Direction-Aware Proximity for Graph Mining KDD 2007, San Jose Hanghang Tong, Yehuda Koren, Christos Faloutsos.
Using Structure Indices for Efficient Approximation of Network Properties Matthew J. Rattigan, Marc Maier, and David Jensen University of Massachusetts.

Using Structure Indices for Efficient Approximation of Network Properties Matthew J. Rattigan, Marc Maier, and David Jensen University of Massachusetts.
© University of Minnesota Data Mining for the Discovery of Ocean Climate Indices 1 CSci 8980: Data Mining (Fall 2002) Vipin Kumar Army High Performance.

© University of Minnesota Data Mining for the Discovery of Ocean Climate Indices 1 CSci 8980: Data Mining (Fall 2002) Vipin Kumar Army High Performance.
Zdravko Markov and Daniel T. Larose, Data Mining the Web: Uncovering Patterns in Web Content, Structure, and Usage, Wiley, 2007. Slides for Chapter 1:

Zdravko Markov and Daniel T. Larose, Data Mining the Web: Uncovering Patterns in Web Content, Structure, and Usage, Wiley, Slides for Chapter 1:
HTL-ACTS Workshop, June 2006, New York City Improving Email Speech Acts Analysis via N-gram Selection Vitor R. Carvalho & William W. Cohen Carnegie Mellon.

HTL-ACTS Workshop, June 2006, New York City Improving Speech Acts Analysis via N-gram Selection Vitor R. Carvalho & William W. Cohen Carnegie Mellon.
Iterative Set Expansion of Named Entities using the Web Richard C. Wang and William W. Cohen Language Technologies Institute Carnegie Mellon University.

Iterative Set Expansion of Named Entities using the Web Richard C. Wang and William W. Cohen Language Technologies Institute Carnegie Mellon University.
Machine Learning for Personal Information Management William W. Cohen Machine Learning Department and Language Technologies Institute School of Computer.

Machine Learning for Personal Information Management William W. Cohen Machine Learning Department and Language Technologies Institute School of Computer.
Language-Independent Set Expansion of Named Entities using the Web Richard C. Wang & William W. Cohen Language Technologies Institute Carnegie Mellon University.

Language-Independent Set Expansion of Named Entities using the Web Richard C. Wang & William W. Cohen Language Technologies Institute Carnegie Mellon University.
Fine-tuning Ranking Models: a two-step optimization approach Vitor Jan 29, 2008 Text Learning Meeting - CMU With invaluable ideas from ….

Fine-tuning Ranking Models: a two-step optimization approach Vitor Jan 29, 2008 Text Learning Meeting - CMU With invaluable ideas from ….
STRUCTURED PERCEPTRON Alice Lai and Shi Zhi. Presentation Outline Introduction to Structured Perceptron ILP-CRF Model Averaged Perceptron Latent Variable.

STRUCTURED PERCEPTRON Alice Lai and Shi Zhi. Presentation Outline Introduction to Structured Perceptron ILP-CRF Model Averaged Perceptron Latent Variable.
Cao et al. ICML 2010 Presented by Danushka Bollegala.

Cao et al. ICML 2010 Presented by Danushka Bollegala.

Similar presentations

Ads by Google