1. Entity-Level Modelling for Coreference Resolution
---Paper Summaries
Dongfang Xu
School of Information, University of Arizona
Dec 1, 2016

2. AGENDA Task Definition Basic Model & Features
Factor Graph & Transitive Model
Objective Function

3. Task Definition The first three mentions are all distinct entities,
Example
The first three mentions are all distinct entities,
theyA and theyB refer to people, and them refers to
art items. (Linguistic Perspective, binding principles)
This observation argues for enforcing agreement of entity-level semantic properties during inference, specifically properties relating to permitted semantic roles.

4. Introduction Paper Summaries Entity-level information used in model.
Discriminative probabilistic model: Factor graph
Mentions manage their partial membership in various coreference chains, information about entity-level properties is decentralized and propagated across individual mentions, and they never need to explicitly instantiate entities.
Target: find coreference links, not pairwise binary classifier.
Performance is competitive with Stanford coreference resolution.
Entity-level information for each mention is used through various coreference chains, not attached with the mention itself, why called decentralized.

5. Basic Factor Graph Model
Basic Model
Basic Factor Graph Model
1. Each mention i has an associated random variable ai taking values in
the set {1, , i−1,<new>}, this variable ai specifies
mention i’s selected antecedent or indicates
that it begins a new coreference chain.

6. Basic Factor Graph Model
Basic Model
Basic Factor Graph Model
2. A1, A2, A3, A4 are Unary factors (only linking with one variable node).
fA(i, ai, x) is a feature function that examines
the coreference decision ai for mention i with
document context x; note that this feature function
can include pairwise features based on mention i
and the chosen antecedent ai in the contex x.

7. Basic Factor Graph Model
Basic Model
Basic Factor Graph Model
3. The Conditional distribution
p(a|x)
4. Given a setting of w, we can
Determine:

8. Features used in basic model
Pairwise Features
Features used in basic model

9. Features used in basic model
Pairwise Features
Features used in basic model
For each of the features they present, two conjoined versions are included: one with an indicator of the type of the current mention being resolved, and one with an indicator of the types of the current and antecedent mentions.

10. Factor Graph
Factor Graph Intro
Directed Graph to Undirected graph

11. Factor Graph Factor Graph Intro
What we care about is marginal distribution, we can get joint distribution first.

12. Factor Graph Factor Graph Intro Rule to calculate the factor graph:
(1) Find the target variable node.
(2) Find the leaf nodes (variable node or factor nodes)
(3) Initialization
(4) Factor nodes to variable node:
(5) Variable Nodes to Factor node:

13. Transitive Model Transitive Model I
Each mention has been augmented with a
single property node. Unary Pi factors
encode prior knowledge about the setting
of each Pi.
Factors may be hard (I will not refer to a plural
entity), soft (such as a distribution over named
entity types output by an NER tagger), or
practically uniform(e.g. the last name Smith does
not specify a particular gender).
Binary factors enforce the agreement of a
particular property.

14. Transitive Model Transitive Model II
Strictly enforcing agreement may not
always be correct.
Organizations and geo-political entities are two frequently confused and ambiguous
tags, and in the gold-standard coreference chains it may be the case that a single chain contains instances
of both.
The notion of raw property values ri ∈ {1, ..., k} together with priors in
the form of the Ri factors.
The Ri are constant factors, and may come
from an upstream model or some other source depending on the property being modeled.

15. Transitive Model Transitive Model II
The notion of raw property values ri ∈ {1, ..., k} together with priors in
the form of the Ri factors.
The Pi factors now have a new structure: they now represent a featurized projection of the ri onto the pi, which can now be thought of as “coreferenced-adapted” properties.
where fP is a feature vector over the projection of ri onto pi.

16. Transitive Model Transitive Model II
l 2{1, 2, 3,…,m} for each of m properties.
where i and j range over mentions, l ranges over each of m properties, and the outer sum indicates marginalization over all p and r variables.

17. AGENDA Task Definition Basic Model & Features
Factor Graph & Transitive Model
Objective Function

18. Objective Function
Transitive Model II

19. Reference
Durrett, G., Hall, D. L. W., & Klein, D. (2013, August). Decentralized Entity-Level Modeling for Coreference Resolution. In ACL (1) (pp ).
Bishop, C. M. (2006). Pattern recognition. Machine Learning, 128.