1. INTRODUCTION TO ARTIFICIAL INTELLIGENCE Massimo Poesio Supervised Relation Extraction

2. RE AS A CLASSIFICATION TASK Binary relations Entities already manually/automatically recognized Examples are generated for all sentences with at least 2 entities Number of examples generated per sentence is NC2 – Combination of N distinct entities selected 2 at a time

3. GENERATING CANDIDATES TO CLASSIFY

4. RE AS A BINARY CLASSIFICATION TASK

5. NUMBER OF CANDIDATES TO CLASSIFY – SIMPLE MINDED VERSION

6. THE SUPERVISED APPROACH TO RE Most current approaches to RE are kernel- based Different information is used – Sequences of words, e.g., through the GLOBAL CONTEXT / LOCAL CONTEXT kernels of Bunescu and Mooney / Giuliano Lavelli & Romano – Syntactic information through the TREE KERNELS of Zelenko et al / Moschitti et al – Semantic information in recent work

7. KERNEL METHODS: A REMINDER Embedding the input data in a feature space Using a linear algorithm for discovering non-linear patterns Coordinates of images are not needed, only pairwise inner products Pairwise inner products can be efficiently computed directly from X using a kernel function K:X×X→R

8. MODULARITY OF KERNEL METHODS

9. THE WORD-SEQUENCE APPROACH Shallow linguistic Information: – tokenization – Lemmatization – sentence splitting – PoS tagging Claudio Giuliano, Alberto Lavelli, and Lorenza Romano (2007), FBK-IRST: Kernel methods for relation extraction, Proc. Of SEMEVAL-2007

10. LINGUISTIC REALIZATION OF RELATIONS Bunescu & Mooney, NIPS 2005

11. WORD-SEQUENCE KERNELS Two families of “basic” kernels – Global Context – Local Context Linear combination of kernels Explicit computation – Extremely sparse input representation

12. THE GLOBAL CONTEXT KERNEL

14. THE LOCAL CONTEXT KERNEL

15. LOCAL CONTEXT KERNEL (2)

16. KERNEL COMBINATION

17. EXPERIMENTAL RESULTS Biomedical data sets – AIMed – LLL Newspaper articles – Roth and Yih SEMEVAL 2007

18. EVALUATION METHODOLOGIES

19. EVALUATION (2)

20. EVALUATION (3)

21. EVALUATION (4)

22. RESULTS ON AIMED

23. OTHER APPROACHES TO RE Using syntactic information Using lexical features

24. Syntactic information for RE Pros: – more structured information useful when dealing with long-distance relations Cons: – not always robust – (and not available for all languages)

25. Zelenko et al JMLR 2003 TREE KERNEL defined over a shallow parse tree representation of the sentences – approach vulnerable to unrecoverable parsing errors data set: 200 news articles (not publicly available) two types of relations : person-affiliation and organization-location

26. ZELENKO ET AL

27. CULOTTA & SORENSEN 2004 generalized version of Zelenko’s kernel based on dependency trees (smallest dependency tree containing the two entities of the relation) a bag-of-words kernel is used to compensate syntactic errors data set: ACE 2002 & 2003 results: syntactic information improves performance w.r.t. bag-of-words (good precision but low recall)

28. CULOTTA AND SORENSEN (2)

29. EVALUATION CAMPAIGNS FOR RE Much of modern evaluation of methods is done by competing with other teams on evaluation campaigns like MUC and ACE Modern evaluation campaigns for RE: SEMEVAL (now *SEM) Interesting to look also at the problems of – DATA CREATION – EVALUATION METRICS

30. SEMEVAL 2007 4th International Workshop on Semantic Evaluations Task 04: Classification of Semantic Relations between Nominals – organizers: Roxana Girju, Marti Hearst, Preslav Nakov, ViviNastase, Stan Szpakowicz, Peter Turney, Deniz Yuret – 14 participating teams

31. SEMEVAL 2007: THE RELATIONS

32. SEMEVAL 2007: DATASET CREATION

33. SEMEVAL 2007: DATASET CREATION (2)

34. SEMEVAL 2007 – DATASET CREATION (3)

35. SEMEVAL 2007 – DATASET CREATION (4)

36. SEMEVAL 2007: DATASET

37. SEMEVAL 2007: COMPETITION

38. SEMEVAL 2007: COMPETITION (2)

39. SEMEVAL 2007: BEST RESULTS

40. INFLUENCE OF NER ON RE

41. INFLUENCE OF NER ON RE (2)

42. GENERATING CANDIDATES

45. ACKNOWLEDGMENTS Many slides borrowed from – Roxana Girju – Alberto Lavelli