1. A Deep Memory Network for Chinese Zero Pronoun Resolution
Qingyu Yin, Yu Zhang, Weinan Zhang and Ting Liu
Research Center for Social Computing and Information Retrieval,
Harbin Institute of Technology, Harbin, China

2. 1
Introduction

3. What is Zero Pronoun Zero Pronoun (ZP) The gap in the sentence
Express the sentence
Represent something that is omitted
Certain entities

4. Introduction Goal of this paper ZP is ubiquitous in Chinese
Recover the ZP in the sentence
Entities ：Antecedent
Noun Phrases
ZP – Antecedents
我吃了一个苹果，<ZP> 很甜。
ZP is ubiquitous in Chinese
Overt Pronoun
96%(English)
64%(Chinese)

5. A common way to ZP resolution
Classification Problem
ZP (我吃了一个苹果，<ZP> 很甜)
Select a set of NP candidates
Mention pair approach
Classify for each pair
ZP,NP1 （鸭梨）
ZP,NP2 （苹果） …
ZP,NPn （小明的书）
ZP – NP2

6. Challenges of ZP resolution
Overlook Semantic information
Difficult to represent ZPs
zero pronoun –overt pronouns
No descriptive information
Gender（男、女）
Number（单数、复数）
Represent gaps with some available components
Context information
<ZP> 很甜

7. Challenges of ZP resolution
Overlook Semantic information
Represent gaps with some available components
Context information
Potential candidate antecedents
Only some subsets of candidate antecedents are needed
Select importance candidates
Utilize them to build up representations for the ZP
Memory Network
Use the importance of candidate explicitly

8. 2
The approach

9. The approach -- ZP Semantic information is overlooked
ZP has no descriptive information
No actual content

10. The approach -- ZP Semantic information is overlooked Represent ZP
ZP has no descriptive information
No actual content
Represent ZP
contextual information
<ZP> taste sweet.
apples
books

11. The approach -- ZP For ZP ZP-centered LSTM Employ two LSTM
one to model the preceding context
one to model the following context

12. The approach – Select NP
Represent NP
Average content
Head word of an NP
LSTM-based approach for modeling NP
Content information
Context information

13. The approach – Memory Network
Memory: { r(np1), r(np2), r(np3), …, r(npn) }
Select NP to fill in the gap (ZP)

14. The approach – Memory Network
Memory: { r(np1), r(np2), r(np3), …, r(npn) }
Select NP to fill in the gap (ZP)

15. The approach – Memory Network
Memory: { r(np1), r(np2), r(np3), …, r(npn) }
Select NP to fill in the gap (ZP)

16. The approach Get attention score for each NP candidate
r(ZP), r(npi)
feature vector
ve(ZP,npi)
si = tanh(W(r(ZP), r(npi) , ve(ZP,npi) ))
For all the candidate NPs
Add a softmax layer to gain the final attention score

17. 3
Experimental results

18. Experimental Results Data set: Ontonotes 5.0 Experimental results R P
Baseline： Chinese zero pronoun resolution: A deep learning approach. [C] ACL. - Chen chen and Ng R P F
Baseline system
51.0
51.4
51.2
Our approach (hop 1)
53.0
53.3
53.1
Our approach (hop 2)
53.7
54.0
53.9
Our approach (hop 3)
54.2
54.1
Our approach (hop 4)
54.4
54.7
54.3

19. Experimental Results Effectiveness of modeling ZPs and NPs R P F
ZPContextFree
52.0
51.7
51.9
AntContextAvg
51.4
51.5
AntContHead
52.2
52.5
52.3
Our approach (hop 1)
53.0
53.3
53.1

20. Experimental Results
Visualize Attention

21. 4
Conclusion

22. Conclusion Effective memory network for modeling ZPs Future work
contextual information
ZP-centered LSTM
candidate antecedents
Modeling candidate antecedents
multi-layer attention
Memory network
Future work
Embeddings
UNK embeddings
Avoid Feature engineering

23. Thanks ! Q&A