1. A Cross-Lingual ILP Solution to Zero Anaphora Resolution Ryu Iida & Massimo Poesio (ACL-HLT 2011)

2. Zero-anaphora resolution  Anaphoric function in which phonetic realization of anaphors is not required in “pro-drop” languages  Based on speaker and hearer’s shared understanding  φ : zero-anaphor (non-realized argument)  Essential: 64.3% of anaphors in Japanese newspaper articles are zeros (Iida et al. 2007) English: John went to visit some friends. On the way, he bought some wine. Italian: Giovanni andò a far visita a degli amici. Per via, φ comprò del vino. Japanese: John-wa yujin-o houmon-sita. Tochu-de φ wain-o ka-tta. 2

3. Research background  Zero-anaphora resolution has remained an active area for Japanese (Seki et al. 2002, Isozaki&Hirao 2003, Iida et al. 2007, Imamura et al. 2009, Sasano et al. 2009, Taira et al. 2010)  The availability of the annotated corpora such that provided by SemEVAL2010 task10 “Multi-lingual coreference (Recasens et al.2010) is leading to renewed interest (e.g. Italian)  Mediocre results obtained on zero anaphors by most systems in SemEVAL e.g. I-BART’s recall on zeros < 10% 3

4. Resolving zero-anaphors requires  The simultaneous decision of  Zero-anaphor detection: find phonetically unrealized arguments of predicates (e.g. verbs)  Antecedent identification: search for an antecedent of a zero-anaphor  Roughly correspond to anaphoricity determination and antecedent identification in coreference resolution  Denis&Baldridge(2007) proposed a solution to optimize the outputs from anaphoricity determination and antecedent identification by using Integer Linear Programming (ILP) 4

5. Main idea  Apply Denis&Baldridge (2007)’s ILP framework to zero-anaphora resolution  Extend the ILP framework into a two-way to make it more suitable for zero-anaphora resolution  Focus on Italian and Japanese zero-anaphora to investigate whether or not our approach is useful across languages  Study only subject zero-anaphors (only type in Italian) 5

6. Topic of contents  Research background  Denis&Baldridge (2007)’s ILP model  Proposal: extending the ILP model  Empirical evaluations  Summary & future directions 6

7. Denis&Baldrige (2007)’s ILP formulation of base model  object function  If, mentions i and j are coreferent and mention j is an anaphor : 1 if mentions i and j are coreferent; otherwise 0 7

8. Denis&Baldrige (2007)’s ILP formulation of joint model  object function  If, mentions i and j are coreferent and mention j is an anaphor; otherwise j is non-anaphoric : 1 if mention j is an anaphor; otherwise 0 8

9. 3 constraints in ILP model characteristics of coreference relations transitivity of coreference chains 1. Resolve only anaphors: if mention pair ij is coreferent, mention j must be anaphoric 2. Resolve anaphors: if mention j is anaphoric, it must be coreferent with at least one antecedent 3. Do not resolve non-anaphors: if mention j is non-anaphoric, it should be have no antecedents 9

10. Proposal: extending the ILP framework  Denis&Baldridge’s original ILP-based model is not suitable for zero-anaphora resolution  Two modifications 1. Applying best-first solution 2. Incorporating a subject detection model 10

11. 1. Best-first solution  Select at most one antecedent for an anaphor  “Do-not-resolve-anaphors” constraint is too weak Allow the redundant choice of more than one candidate antecedent Lead to decreasing precision on zero-anaphora resolution  “Do-not-resolve-anaphors” constraint is replaced with “Best First constraint (BF)” that blocks selection of more than one antecedent: 11

12. 2. Integrating subject detection model  Zero-anaphor detection  Difficulty in zero-anaphora resolution comparing to pronominal reference resolution  Simply relying on the parser is not enough  most dependency parsers are not very accurate at identifying grammatical roles  detecting subject is crucial for zero-anaphor detection 12

13. 2. Integrating subject detection model  Resolve only non-subjects: if a predicate j syntactically depends on a subject, the predicate j should have no antecedent of its zero anaphor : 1 if predicate j syntactically depends on a subject; otherwise 0 13

14. Experiment 1: zero-anaphors  Compare the baseline models with the extended ILP-based models  Use the Maximum Entropy model to create base classifiers in the ILP framework and baselines  Feature definitions basically follow the previous work (Iida et al. 2007) and (Poesio et al. 2010) 14

15. Two baseline models  PAIRWISE classification model (PAIRWISE)  Antecedent identification and anaphoricity determination are simultaneously executed by a single classifier (as in Soon et al. 2001)  Anaphoricity Determination-then-Search antecedent CASCADEd model (DS-CASCADE) 1. Filter out non-anaphoric candidate anaphors using an anaphoricity determination model 2. Select an antecedent from a set of candidate antecedents of anaphoric anaphors using an antecedent identification model 15

16. Data sets  Italian (Wikipedia articles)  LiveMemories text corpus 1.2 (Rodriguez et al. 2010) Data set on the SemEval2010: Coreference Resolution in Multiple Languages #zero-anaphors: train 1,160 / test 837  Japanese (newspaper articles)  NAIST text corpus (Iida et al. 2007) ver.1.4ß #zero-anaphors: train 29,544 / test 11,205 16

17. Creating subject detection models  Data sets  Italian: 80,878 tokens in TUT corpus (Bosco et al. 2010)  Japanese: 1753 articles (i.e. training dataset) in NAIST text corpus merged with Kyoto text corpus dependency arc is judged as positive if its relation is subject; as negative otherwise  Induce a maximum entropy classifier based on the labeled arcs  Features  Italian: lemmas, PoS tags and morphological information automatically computed by TextPro (Pianta et al. 2008)  Japanese: similar features as Italian except gender and number information 17

18. Results for zero anaphors ItalianJapanese modelRPFRPF PAIRWISE0.8640.1720.2870.2860.3080.296 DS-CASCADE0.3960.6840.5020.3450.1940.248 ILP0.9050.0340.0650.3790.2380.293 ILP+BF0.8030.3750.5110.3530.2560.297 ILP+SUBJ0.9000.0340.0660.3710.3150.341 ILP+BF+SUBJ0.7770.3980.5260.3450.3480.346 +BF: use best first constraint, +SUBJ: use subject detection model 18

19. Experiment 2: all anaphors 19  Investigate performance of all anaphors (i.e. NP- coreference and zero-anaphors)  Use the same data set and same data separation  Italian: LiveMemories text corpus 1.2  Japanese: NAIST text corpus 1.4ß  Performance of each model are compared in terms of MUC score  Different types of referring expressions display very different anaphoric behavior  Induce 2 different models for NP-coreference and zero-anaphora respectively

20. ItalianJapanese modelRPFRPF PAIRWISE0.5660.3140.4040.4270.2400.308 DS-CASCADE0.2460.6860.3620.2910.4880.365 I-BART (Poesio et al. 2010) 0.5320.4410.482--- ILP0.6070.3840.4700.4900.3040.375 ILP+BF0.5630.5190.5400.4460.3400.386 ILP+SUBJ0.6060.3870.4730.4840.3530.408 ILP+BF+SUBJ0.5590.5360.5470.4410.4150.427 Results for all anaphors 20

21. Summary  Extended Denis&Baldridge (2007)’s ILP-based coreference resolution model by incorporating modified constraints & a subject detection model  Our results show the proposed model obtained improvement on both zero-anaphora resolution and overall coreference resolution 21

22. Future directions  Introduce more sophisticated antecedent identification model  Test our model for English constructions resembling zero-anaphora  Null instantiations in SEMEVAL 2010 ‘Linking Events and their Participants in Discourse’ task  Detect generic zero-anaphors  Have no antecedent in the preceding context  e.g. the Italian and Japanese translation of I walked into the hotel and (they) said … 22

23. 23

24. Data sets on English coreference  Use ACE-2002 data set  Data set is classified into the two subset  Pronouns and NPs 24

25. Details of experiment: English 25 training data train: NPs train: zeros models: NP coreference models: zero anaphora test data test: NPs test: zeros outputs: all anaphors outputs: NPs outputs: zeros

26. Results: all anaphors (English) 26 English modelRPF PAIRWISE0.6390.6750.656 DS-CASCADE0.597 ILP0.7360.3800.501 ILP+BF0.6650.7140.689 ILP+SUBJ--- ILP+BF+SUBJ---