Easy-First Coreference Resolution Veselin Stoyanov and Jason Eisner Johns Hopkins University

Coreference Resolution Given a peace of text, identify all noun phrases that refer to the same real-world entity

Coreference Resolution Given a peace of text, identify all noun phrases that refer to the same real-world entity Former Boxing Champion Mike Tyson revealed that he had caught Hollywood heartthrob Brad Pitt with his ex-wife Robin Givens. In an interview Tyson said that he was amidst the process of divorce from Givens in the late 1980s when he found her sleeping with the Hollywood star.

Coreference Resolution Given a peace of text, identify all noun phrases that refer to the same real-world entity Former Boxing Champion Mike Tyson revealed that he had caught Hollywood heartthrob Brad Pitt with his ex-wife Robin Givens. In an interview Tyson said that he was amidst the process of divorce from Givens in the late 1980s when he found her sleeping with the Hollywood star.

Coreference Resolution Given a peace of text, identify all noun phrases that refer to the same real-world entity Former Boxing Champion Mike Tyson revealed that he had caught Hollywood heartthrob Brad Pitt with his ex-wife Robin Givens. In an interview Tyson said that he was amidst the process of divorce from Givens in the late 1980s when he found her sleeping with the Hollywood star.

Coreference Resolution Typical pairwise approaches But, inherently global task Former Boxing Champion Mike Tyson revealed that he had caught Hollywood heartthrob Brad Pitt with his ex-wife Robin Givens. In an interview Tyson said that he was amidst the process of divorce from Givens in the late 1980s when he found her sleeping with the Hollywood star.

Coreference Resolution Inherently global task Former Boxing Champion Mike Tyson revealed that he had caught Hollywood heartthrob Brad Pitt with his ex-wife Robin Givens. In an interview Tyson said that he was amidst the process of divorce from Givens in the late 1980s when he found her sleeping with the Hollywood star.

Coreference Resolution Inherently global task Former Boxing Champion Mike Tyson revealed that he had caught Hollywood heartthrob Brad Pitt with his ex-wife Robin Givens. In an interview Tyson said that he was amidst the process of divorce from Givens in the late 1980s when he found her sleeping with the Hollywood star.

Global Approaches to Coreference Resolution A difficult search problem – Exponentially many possible clusterings Recent approaches have been successful: – [Culotta et al., 2007] – train a global coherence measure; use greedy inference at test time – [Poon and Domingos, 2008] – use an unsupervised approach based on MLNs to enforce global properties

A Different Approach A Multi-Pass Sieve [Raghunathan et al., 2010] – Build coreference clusters incrementally – Applying sieves (hand-specified rules) of decreasing precision and increasing recall – Winner of the CoNLL-2011 shared task

A Multi-Pass Sieve for Coreference Resolution [Raghunathan et al. 2010] Former Boxing Champion Mike Tyson revealed that he had caught Hollywood heartthrob Brad Pitt with his ex-wife Robin Givens. In an interview Tyson said that he was amidst the process of divorce from Givens in the late 1980s when he found her sleeping with the Hollywood star.

A Multi-Pass Sieve for Coreference Resolution [Raghunathan et al. 2010] Sieves: 1.Exact match Former Boxing Champion Mike Tyson revealed that he had caught Hollywood heartthrob Brad Pitt with his ex-wife Robin Givens. In an interview Tyson said that he was amidst the process of divorce from Givens in the late 1980s when he found her sleeping with the Hollywood star.

A Multi-Pass Sieve for Coreference Resolution [Raghunathan et al. 2010] Sieves: 1.Exact match 2.Precise Constructs Former Boxing Champion Mike Tyson revealed that he had caught Hollywood heartthrob Brad Pitt with his ex-wife Robin Givens. In an interview Tyson said that he was amidst the process of divorce from Givens in the late 1980s when he found her sleeping with the Hollywood star.

A Multi-Pass Sieve for Coreference Resolution [Raghunathan et al. 2010] Sieves: 1.Exact match 2.Precise Constructs Former Boxing Champion Mike Tyson revealed that he had caught Hollywood heartthrob Brad Pitt with his ex-wife Robin Givens. In an interview Tyson said that he was amidst the process of divorce from Givens in the late 1980s when he found her sleeping with the Hollywood star.

A Multi-Pass Sieve for Coreference Resolution [Raghunathan et al. 2010] Sieves: 1.Exact match 2.Precise Constructs 3.Strict Head match Former Boxing Champion Mike Tyson revealed that he had caught Hollywood heartthrob Brad Pitt with his ex-wife Robin Givens. In an interview Tyson said that he was amidst the process of divorce from Givens in the late 1980s when he found her sleeping with the Hollywood star.

A Multi-Pass Sieve for Coreference Resolution [Raghunathan et al. 2010] Sieves: 1.Exact match 2.Precise Constructs 3.Strict Head match Former Boxing Champion Mike Tyson revealed that he had caught Hollywood heartthrob Brad Pitt with his ex-wife Robin Givens. In an interview Tyson said that he was amidst the process of divorce from Givens in the late 1980s when he found her sleeping with the Hollywood star.

A Multi-Pass Sieve for Coreference Resolution [Raghunathan et al. 2010] Sieves: 1.Exact match 2.Precise Constructs 3.Strict Head match …………. 7. Pronoun Matching Former Boxing Champion Mike Tyson revealed that he had caught Hollywood heartthrob Brad Pitt with his ex-wife Robin Givens. In an interview Tyson said that he was amidst the process of divorce from Givens in the late 1980s when he found her sleeping with the Hollywood star.

A Multi-Pass Sieve for Coreference Resolution [Raghunathan et al. 2010] Sieves: 1.Exact match 2.Precise Constructs 3.Strict Head match …………. 7. Pronoun Matching Former Boxing Champion Mike Tyson revealed that he had caught Hollywood heartthrob Brad Pitt with his ex-wife Robin Givens. In an interview Tyson said that he was amidst the process of divorce from Givens in the late 1980s when he found her sleeping with the Hollywood star.

Easy-First Coreference Resolution Use the same idea: – Greedily build clusters – Using more information for the less confident decisions But learn automatically what constitutes an “easy” (confident) decision – Leads to empirical improvement over the Multi- Pass Sieve approach and a competitive baseline

Easy-First Coreference Resolution Incrementally and greedily build clusters – Using machine learning Learn policy for the test time inference procedure

Easy-First Coreference: Talk Overview Test-Time Inference Training Algorithm Experimental results

TEST-TIME INFERENCE

Test-Time Inference Begin with each NP in a separate cluster Former Boxing Champion Mike Tyson revealed that he had caught Hollywood heartthrob Brad Pitt with his ex-wife Robin Givens. In an interview Tyson said that he was amidst the process of divorce from Givens in the late 1980s when he found her sleeping with the Hollywood star.

Test-Time Inference Former Boxing Champion Mike Tyson revealed that he had caught Hollywood heartthrob Brad Pitt with his ex-wife Robin Givens. In an interview Tyson said that he was amidst the process of divorce from Givens in the late 1980s when he found her sleeping with the Hollywood star. Cluster 1 SemType: uknwn Gender: uknwn Number: single Names: NIL Cluster 2 SemType: person Gender: male Number: single Names: Mike Tyson Cluster 3 SemType: person Gender: male Number: single Names: NIL Cluster 3 SemType: person Gender: male Number: single Names: NIL Begin with each NP in a separate cluster

Compute a feature vector for each pair of clusters Test-Time Inference Former Boxing Champion Mike Tyson revealed that he had caught Hollywood heartthrob Brad Pitt with his ex-wife Robin Givens. In an interview Tyson said that he was amidst the process of divorce from Givens in the late 1980s when he found her sleeping with the Hollywood star. Cluster 1 SemType: uknwn Gender: uknwn Number: single Names: NIL Cluster 2 SemType: person Gender: male Number: single Names: Mike Tyson

Compute a feature vector for each pair of clusters Test-Time Inference Former Boxing Champion Mike Tyson revealed that he had caught Hollywood heartthrob Brad Pitt with his ex-wife Robin Givens. In an interview Tyson said that he was amidst the process of divorce from Givens in the late 1980s when he found her sleeping with the Hollywood star. Cluster 1 SemType: uknwn Gender: uknwn Number: single Names: NIL Cluster 2 SemType: person Gender: male Number: single Names: Mike Tyson SemTypeSame = False

Compute a feature vector for each pair of clusters Test-Time Inference Former Boxing Champion Mike Tyson revealed that he had caught Hollywood heartthrob Brad Pitt with his ex-wife Robin Givens. In an interview Tyson said that he was amidst the process of divorce from Givens in the late 1980s when he found her sleeping with the Hollywood star. Cluster 1 SemType: uknwn Gender: uknwn Number: single Names: NIL Cluster 2 SemType: person Gender: male Number: single Names: Mike Tyson SemTypeSame = False SemTypeComp = True

Compute a feature vector for each pair of clusters Test-Time Inference Former Boxing Champion Mike Tyson revealed that he had caught Hollywood heartthrob Brad Pitt with his ex-wife Robin Givens. In an interview Tyson said that he was amidst the process of divorce from Givens in the late 1980s when he found her sleeping with the Hollywood star. Cluster 1 SemType: uknwn Gender: uknwn Number: single Names: NIL Cluster 2 SemType: person Gender: male Number: single Names: Mike Tyson SemTypeSame = False SemTypeComp = True NumberSame = True

Compute a feature vector for each pair of clusters Test-Time Inference Former Boxing Champion Mike Tyson revealed that he had caught Hollywood heartthrob Brad Pitt with his ex-wife Robin Givens. In an interview Tyson said that he was amidst the process of divorce from Givens in the late 1980s when he found her sleeping with the Hollywood star. Cluster 1 SemType: uknwn Gender: uknwn Number: single Names: NIL Cluster 2 SemType: person Gender: male Number: single Names: Mike Tyson SemTypeSame = False SemTypeComp = True NumberSame = True RoleAppositive = True

Compute a feature vector for each pair of clusters Test-Time Inference Former Boxing Champion Mike Tyson revealed that he had caught Hollywood heartthrob Brad Pitt with his ex-wife Robin Givens. In an interview Tyson said that he was amidst the process of divorce from Givens in the late 1980s when he found her sleeping with the Hollywood star. Cluster 1 SemType: uknwn Gender: uknwn Number: single Names: NIL Cluster 2 SemType: person Gender: male Number: single Names: Mike Tyson SemTypeSame = False SemTypeComp = True NumberSame = True RoleAppositive = True SentDistance = 0

Compute a feature vector for each pair of clusters Test-Time Inference Former Boxing Champion Mike Tyson revealed that he had caught Hollywood heartthrob Brad Pitt with his ex-wife Robin Givens. In an interview Tyson said that he was amidst the process of divorce from Givens in the late 1980s when he found her sleeping with the Hollywood star. Cluster 1 SemType: uknwn Gender: uknwn Number: single Names: NIL Cluster 2 SemType: person Gender: male Number: single Names: Mike Tyson SemTypeSame = False SemTypeComp = True NumberSame = True RoleAppositive = True SentDistance = 0 Global features Local features

Compute a feature vector for each pair of clusters Test-Time Inference Former Boxing Champion Mike Tyson revealed that he had caught Hollywood heartthrob Brad Pitt with his ex-wife Robin Givens. In an interview Tyson said that he was amidst the process of divorce from Givens in the late 1980s when he found her sleeping with the Hollywood star. Cluster 1 SemType: uknwn Gender: uknwn Number: single Names: NIL Cluster 2 SemType: person Gender: male Number: single Names: Mike Tyson f(join ij )

Compute a weight for each join action Test-Time Inference Former Boxing Champion Mike Tyson revealed that he had caught Hollywood heartthrob Brad Pitt with his ex-wife Robin Givens. In an interview Tyson said that he was amidst the process of divorce from Givens in the late 1980s when he found her sleeping with the Hollywood star. Cluster 1 SemType: uknwn Gender: uknwn Number: single Names: NIL Cluster 2 SemType: person Gender: male Number: single Names: Mike Tyson w.f(join ij )

Compute a weight for each join action Test-Time Inference Former Boxing Champion Mike Tyson revealed that he had caught Hollywood heartthrob Brad Pitt with his ex-wife Robin Givens. In an interview Tyson said that he was amidst the process of divorce from Givens in the late 1980s when he found her sleeping with the Hollywood star. Cluster 1 SemType: uknwn Gender: uknwn Number: single Names: NIL Cluster 2 SemType: person Gender: male Number: single Names: Mike Tyson w.f(join ij ) weight ij =

Perform the action with the highest weight Test-Time Inference Former Boxing Champion Mike Tyson revealed that he had caught Hollywood heartthrob Brad Pitt with his ex-wife Robin Givens. In an interview Tyson said that he was amidst the process of divorce from Givens in the late 1980s when he found her sleeping with the Hollywood star. Cluster 1 SemType: uknwn Gender: uknwn Number: single Names: NIL Cluster 2 SemType: person Gender: male Number: single Names: Mike Tyson w.f(join ij ) a ij =argmax ij

Perform the action with the highest weight Test-Time Inference Former Boxing Champion Mike Tyson revealed that he had caught Hollywood heartthrob Brad Pitt with his ex-wife Robin Givens. In an interview Tyson said that he was amidst the process of divorce from Givens in the late 1980s when he found her sleeping with the Hollywood star. Cluster 1 SemType: uknwn Gender: uknwn Number: single Names: NIL Cluster 2 SemType: person Gender: male Number: single Names: Mike Tyson w.f(join ij ) a 12 =argmax ij

Perform the action with the highest weight Test-Time Inference Former Boxing Champion Mike Tyson revealed that he had caught Hollywood heartthrob Brad Pitt with his ex-wife Robin Givens. In an interview Tyson said that he was amidst the process of divorce from Givens in the late 1980s when he found her sleeping with the Hollywood star. Cluster 15 SemType: person Gender: male Number: single Names: Mike Tyson w.f(join ij ) a 12 =argmax ij

Perform the action with the highest weight Test-Time Inference Former Boxing Champion Mike Tyson revealed that he had caught Hollywood heartthrob Brad Pitt with his ex-wife Robin Givens. In an interview Tyson said that he was amidst the process of divorce from Givens in the late 1980s when he found her sleeping with the Hollywood star.

Perform the action with the highest weight Test-Time Inference Former Boxing Champion Mike Tyson revealed that he had caught Hollywood heartthrob Brad Pitt with his ex-wife Robin Givens. In an interview Tyson said that he was amidst the process of divorce from Givens in the late 1980s when he found her sleeping with the Hollywood star.

Perform the action with the highest weight Test-Time Inference Former Boxing Champion Mike Tyson revealed that he had caught Hollywood heartthrob Brad Pitt with his ex-wife Robin Givens. In an interview Tyson said that he was amidst the process of divorce from Givens in the late 1980s when he found her sleeping with the Hollywood star.

Perform the action with the highest weight Test-Time Inference Former Boxing Champion Mike Tyson revealed that he had caught Hollywood heartthrob Brad Pitt with his ex-wife Robin Givens. In an interview Tyson said that he was amidst the process of divorce from Givens in the late 1980s when he found her sleeping with the Hollywood star.

Perform the action with the highest weight Test-Time Inference Former Boxing Champion Mike Tyson revealed that he had caught Hollywood heartthrob Brad Pitt with his ex-wife Robin Givens. In an interview Tyson said that he was amidst the process of divorce from Givens in the late 1980s when he found her sleeping with the Hollywood star.

Perform the action with the highest weight Test-Time Inference Former Boxing Champion Mike Tyson revealed that he had caught Hollywood heartthrob Brad Pitt with his ex-wife Robin Givens. In an interview Tyson said that he was amidst the process of divorce from Givens in the late 1980s when he found her sleeping with the Hollywood star.

Perform the action with the highest weight Test-Time Inference Former Boxing Champion Mike Tyson revealed that he had caught Hollywood heartthrob Brad Pitt with his ex-wife Robin Givens. In an interview Tyson said that he was amidst the process of divorce from Givens in the late 1980s when he found her sleeping with the Hollywood star.

Perform the action with the highest weight Test-Time Inference Former Boxing Champion Mike Tyson revealed that he had caught Hollywood heartthrob Brad Pitt with his ex-wife Robin Givens. In an interview Tyson said that he was amidst the process of divorce from Givens in the late 1980s when he found her sleeping with the Hollywood star.

Perform the action with the highest weight Test-Time Inference Former Boxing Champion Mike Tyson revealed that he had caught Hollywood heartthrob Brad Pitt with his ex-wife Robin Givens. In an interview Tyson said that he was amidst the process of divorce from Givens in the late 1980s when he found her sleeping with the Hollywood star.

Perform the action with the highest weight Test-Time Inference Former Boxing Champion Mike Tyson revealed that he had caught Hollywood heartthrob Brad Pitt with his ex-wife Robin Givens. In an interview Tyson said that he was amidst the process of divorce from Givens in the late 1980s when he found her sleeping with the Hollywood star.

Perform the action with the highest weight Test-Time Inference Former Boxing Champion Mike Tyson revealed that he had caught Hollywood heartthrob Brad Pitt with his ex-wife Robin Givens. In an interview Tyson said that he was amidst the process of divorce from Givens in the late 1980s when he found her sleeping with the Hollywood star.

Until the action with the highest weight is the special action HALT Test-Time Inference Former Boxing Champion Mike Tyson revealed that he had caught Hollywood heartthrob Brad Pitt with his ex-wife Robin Givens. In an interview Tyson said that he was amidst the process of divorce from Givens in the late 1980s when he found her sleeping with the Hollywood star.

Time complexity Let n = the number of mentions in a document At most n clusters at any time s = number of joins performed by the algorithm s<=n at most n joins before HALT (usually much less)

Time complexity Each step requires: – Popping the top action -- O(logn) (priority queue) – Performing the join – O(1) – Deleting O(n) old actions – O(nlogn) – Computing scores for O(n) new actions -- O(n) – Inserting those -- O(nlogn) or O(n) Total runtime = O(n 2 logn)+O(snlogn)

TRAINING ALGORITHM

Learning Learn weight vector w that leads the Easy-First algorithm to good clusterings At each step we are given a state s and a set of possible actions {join ij } ∪ {HALT} We want to learn what is “the best” action in each state But the current state s depends on the action taken so far

Reinforcement Learning A classic reinforcement learning setting Act in the world and learn “good” actions in each state We tried several reinforcement learning algorithms – Policy gradient [Sutton et al., 2000] – Q-learning [Watkins and Dayan, 1992] – No success: distant supervision; too many possible states

Perceptron-Style Training A version of the early update structured perceptron [Collins and Roark, 2004] – Inspired by the success of easy-first parsing [Goldberg and Elhadad, 2010] The algorithm: – Keep acting in the world until the next action will result in an error (reduction in score) – Update weights away from the top action and towards the highest scoring positive action

Training algorithm Perform N iterations: – For each document d in the training set do: Initialize C so that each mention in its own cluster Initialize A = {join ij } ∪ {HALT} repeat until a max != HALT: – for k steps do: » a max =A.peekMax() » if a max.isPositive() break » a pos =A.peekMaxPositive() » w+=(feat(a pos,C)-feat(a max,C)) » Recompute scores in A using new w update weight

Training algorithm Perform N iterations: – For each document d in the training set do: Initialize C so that each mention in its own cluster Initialize A = {join ij } ∩ {HALT} repeat until a max != HALT: – Update weights – a max =A.popMax() – if a max is a join » perform a max

EXPERIMENTAL EVALUATION

Experimental Setup Five datasets: – MUC6, MUC7, ACE03, ACE04, ACE05 Two evaluation metrics: – MUC score – B^3 score Implemented in the Reconcile research platform [Stoyanov et al. 2010] – Code available upon request Two settings: – Automatic mentions – Gold (manual) mentions

Baselines A pairwise baseline [Stoyanov et al. 2009] Reimplementation of the pairwise baseline using different features Easy-First coreference resolution – Easy-First percep – weights from the pairwise baseline – Easy-First struct – weights from our training algorithm Only for the gold mentions data – Entity-centered approach [Haghighi and Klein, 2009] – Multi-pass sieve [Raghunathan et al., 2010]

Results: Automatic Mentions MUC6MUC7ACE03ACE04ACE05 B^3 MUCB^3 MUCB^3 MUCB^3 MUCB^3 MUC Reconcile

Results: Automatic Mentions MUC6MUC7ACE03ACE04ACE05 B^3 MUCB^3 MUCB^3 MUCB^3 MUCB^3 MUC Reconcile Pairwise baseline

Results: Automatic Mentions MUC6MUC7ACE03ACE04ACE05 B^3 MUCB^3 MUCB^3 MUCB^3 MUCB^3 MUC Reconcile Pairwise baseline

Results: Automatic Mentions MUC6MUC7ACE03ACE04ACE05 B^3 MUCB^3 MUCB^3 MUCB^3 MUCB^3 MUC Reconcile Pairwise baseline

Results: Automatic Mentions MUC6MUC7ACE03ACE04ACE05 B^3 MUCB^3 MUCB^3 MUCB^3 MUCB^3 MUC Reconcile Pairwise baseline Easy-First percep

Results: Automatic Mentions MUC6MUC7ACE03ACE04ACE05 B^3 MUCB^3 MUCB^3 MUCB^3 MUCB^3 MUC Reconcile Pairwise baseline Easy-First percep Easy-First struct

Results: Automatic Mentions MUC6MUC7ACE03ACE04ACE05 B^3 MUCB^3 MUCB^3 MUCB^3 MUCB^3 MUC Reconcile Pairwise baseline Easy-First percep Easy-First struct

Results: Automatic Mentions MUC6MUC7ACE03ACE04ACE05 B^3 MUCB^3 MUCB^3 MUCB^3 MUCB^3 MUC Reconcile Pairwise baseline Easy-First percep Easy-First struct

Results: Gold Mentions MUC6ACE04 B^3MUCB^3MUC Haghighi and Klein Raghunathan et al

Results: Gold Mentions MUC6ACE04 B^3MUCB^3MUC Haghighi and Klein Raghunathan et al Stoyanov et al

Results: Gold Mentions MUC6ACE04 B^3MUCB^3MUC Haghighi and Klein Raghunathan et al Stoyanov et al

Results: Gold Mentions MUC6ACE04 B^3MUCB^3MUC Haghighi and Klein Raghunathan et al Stoyanov et al

Results: Gold Mentions MUC6ACE04 B^3MUCB^3MUC Haghighi and Klein Raghunathan et al Stoyanov et al Pairwise baseline

Results: Gold Mentions MUC6ACE04 B^3MUCB^3MUC Haghighi and Klein Raghunathan et al Stoyanov et al Pairwise baseline

Results: Gold Mentions MUC6ACE04 B^3MUCB^3MUC Haghighi and Klein Raghunathan et al Stoyanov et al Pairwise baseline Easy-First percep

Results: Gold Mentions MUC6ACE04 B^3MUCB^3MUC Haghighi and Klein Raghunathan et al Stoyanov et al Pairwise baseline Easy-First percep Easy-First struct

Conclusions A new approach to coreference using greedy incremental clustering – More accurate clustering by utilizing global information as it becomes available for less confident decisions Learning policy for the test-time inference procedure leads to improvements

THANK YOU!