UNED at PASCAL RTE-2 Challenge IR&NLP Group at UNED nlp.uned.es Jesús Herrera Anselmo Peñas Álvaro Rodrigo Felisa Verdejo.

Slides:

Advertisements

Similar presentations

Conclusions 101 Wrapping it all up with style!!!.

Advertisements

Distinción semántica de compuestos léxicos en Recuperación de Información Anselmo Peñas, Julio Gonzalo y Felisa Verdejo Dpto. Lenguajes y Sistemas Informáticos,

COGEX at the Second RTE Marta Tatu, Brandon Iles, John Slavick, Adrian Novischi, Dan Moldovan Language Computer Corporation April 10 th, 2006.

COGEX at the Second RTE Marta Tatu, Brandon Iles, John Slavick, Adrian Novischi, Dan Moldovan Language Computer Corporation April 10 th, 2006.

Question Answering for Machine Reading Evaluation Evaluation Campaign at CLEF 2011 Anselmo Peñas (UNED, Spain) Eduard Hovy (USC-ISI, USA) Pamela Forner.

Recognizing Textual Entailment Challenge PASCAL Suleiman BaniHani.

Baselines for Recognizing Textual Entailment Ling 541 Final Project Terrence Szymanski.

Unsupervised Acquisition of Axioms to Paraphrase Noun Compounds and Genitives CICLING 2012, New Delhi Anselmo Peñas NLP & IR Group, UNED, Spain Ekaterina.

Robust Textual Inference via Graph Matching Aria Haghighi Andrew Ng Christopher Manning.

Semantic Entailment Nathaniel Story Ginger Buckbee Greg Lorge Billy Dean.

Recognizing Textual Entailment Progress towards RTE 4 Scott Settembre University at Buffalo, SNePS Research Group

Breadth First Search

Normalized alignment of dependency trees for detecting textual entailment Erwin Marsi & Emiel Krahmer Tilburg University Wauter Bosma & Mariët Theune University.

Predicting Text Quality for Scientific Articles Annie Louis University of Pennsylvania Advisor: Ani Nenkova.

Textual Entailment Using Univariate Density Model and Maximizing Discriminant Function “Third Recognizing Textual Entailment Challenge 2007 Submission”

SRL using complete syntactic analysis Mihai Surdeanu and Jordi Turmo TALP Research Center Universitat Politècnica de Catalunya.

3rd Answer Validation Exercise ( AVE 2008) QA subtrack at Cross-Language Evaluation Forum 2008 UNED Anselmo Peñas Álvaro Rodrigo Felisa Verdejo Thanks.

1 RTE April 10, 2006 An approach based on Logic Forms and WordNet relationships to Textual Entailment performance O. Ferrández, R. M. Terol, R. Muñoz,

Automatic Classification of Semantic Relations between Facts and Opinions Koji Murakami, Eric Nichols, Junta Mizuno, Yotaro Watanabe, Hayato Goto, Megumi.

Using Maximal Embedded Subtrees for Textual Entailment Recognition Sophia Katrenko & Pieter Adriaans Adaptive Information Disclosure project Human Computer.

Third Recognizing Textual Entailment Challenge Potential SNeRG Submission.

ResPubliQA IR baselines and UNED participation Álvaro Rodrigo Joaquín Pérez Anselmo Peñas Guillermo Garrido Lourdes Araujo nlp.uned.es.

Article by: Feiyu Xu, Daniela Kurz, Jakub Piskorski, Sven Schmeier Article Summary by Mark Vickers.

Page 1 Relation Alignment for Textual Entailment Recognition Department of Computer Science University of Illinois at Urbana-Champaign Mark Sammons, V.G.Vinod.

Outline P1EDA’s simple features currently implemented –And their ablation test Features we have reviewed from Literature –(Let’s briefly visit them) –Iftene’s.

Answer Validation Exercise - AVE QA subtrack at Cross-Language Evaluation Forum 2007 UNED (coord.) Anselmo Peñas Álvaro Rodrigo Valentín Sama Felisa Verdejo.

Lecture 9. Arithmetic and geometric series and mathematical induction

Hang Cui et al. NUS at TREC-13 QA Main Task 1/20 National University of Singapore at the TREC- 13 Question Answering Main Task Hang Cui Keya Li Renxu Sun.

The Research Paper: Developing an Outline

“How much context do you need?” An experiment about context size in Interactive Cross-language Question Answering B. Navarro, L. Moreno-Monteagudo, E.

The Research Paper: Developing an Outline

Knowledge and Tree-Edits in Learnable Entailment Proofs Asher Stern, Amnon Lotan, Shachar Mirkin, Eyal Shnarch, Lili Kotlerman, Jonathan Berant and Ido.

Adaptor Grammars Ehsan Khoddammohammadi Recent Advances in Parsing Technology WS 2012/13 Saarland University 1.

RTE Planning Session Luisa Bentivogli, Peter Clark, Ido Dagan, Hoa Trang Dang, Danilo Giampiccolo.

Answer Validation Exercise - AVE QA subtrack at Cross-Language Evaluation Forum UNED (coord.) Anselmo Peñas Álvaro Rodrigo Valentín Sama Felisa Verdejo.

1 Efficient Search Ranking in Social Network ACM CIKM2007 Monique V. Vieira, Bruno M. Fonseca, Rodrigo Damazio, Paulo B. Golgher, Davi de Castro Reis,

2010/2/4Yi-Ting Huang Pennacchiotti, M., & Zanzotto, F. M. Learning Shallow Semantic Rules for Textual Entailment. Recent Advances in Natural Language.

Why is Computational Linguistics Not More Used in Search Engine Technology? John Tait University of Sunderland, UK.

On a Network Creation Game PoA Seminar Presenting: Oren Gilon Based on an article by Fabrikant et al 1.

Ch 1.4 – Equations & Inequalities

Detecting compositionality using semantic vector space models based on syntactic context Guillermo Garrido and Anselmo Peñas NLP & IR Group at UNED Madrid,

Summarization Focusing on Polarity or Opinion Fragments in Blogs Yohei Seki Toyohashi University of Technology Visiting Scholar at Columbia University.

LECTURE 2: SEMANTICS IN LINGUISTICS

Relation Alignment for Textual Entailment Recognition Cognitive Computation Group, University of Illinois Experimental ResultsTitle Mark Sammons, V.G.Vinod.

SEMANTICS An Introduction to Linguistics. What does semantics study? Semantics studies the meaning of language.

WordNet Enhancements: Toward Version 2.0 WordNet Connectivity Derivational Connections Disambiguated Definitions Topical Connections.

QA Pilot Task at CLEF 2004 Jesús Herrera Anselmo Peñas Felisa Verdejo UNED NLP Group Cross-Language Evaluation Forum Bath, UK - September 2004.

Evaluating Answer Validation in multi- stream Question Answering Álvaro Rodrigo, Anselmo Peñas, Felisa Verdejo UNED NLP & IR group nlp.uned.es The Second.

Toward an Open Source Textual Entailment Platform (Excitement Project) Bernardo Magnini (on behalf of the Excitement consortium) 1 STS workshop, NYC March.

Answer Mining by Combining Extraction Techniques with Abductive Reasoning Sanda Harabagiu, Dan Moldovan, Christine Clark, Mitchell Bowden, Jown Williams.

1 Measuring the Semantic Similarity of Texts Author ： Courtney Corley and Rada Mihalcea Source ： ACL-2005 Reporter ： Yong-Xiang Chen.

SALSA-WS 09/05 Approximating Textual Entailment with LFG and FrameNet Frames Aljoscha Burchardt, Anette Frank Computational Linguistics Department Saarland.

Cheap and Fast – But is it Good? Evaluating Non-Expert Annotations for Natural Language Tasks EMNLP 2008 Rion Snow CS Stanford Brendan O’Connor Dolores.

4.2 Area Definition of Sigma Notation = 14.

2-3 Biconditionals and Definitions Objective: To write biconditionals and recognize good definitions.

Learning Event Durations from Event Descriptions Feng Pan, Rutu Mulkar, Jerry R. Hobbs University of Southern California ACL ’ 06.

Overview of Statistical NLP IR Group Meeting March 7, 2006.

Outline of Research Paper Definition:  An outline is a general plan of the material that is to be presented in a speech or a paper.

Pre-processing Tasks for Rule- Based English-Korean Machine Translation System Sung-Dong Kim, Dept. of Computer Engineering, Hansung University, Seoul,

Recognising Textual Entailment Johan Bos School of Informatics University of Edinburgh Scotland,UK.

Writing Inspirations, 2017 Aalto University

Alessandro Lenci University of Pisa

Learning Textual Entailment from Examples

Language, Logic, and Meaning

Writing Inspirations, Spring 2016 Aalto University

Separation of powers.

Recognizing Partial Textual Entailment

Entailment summary Possible to predict when some sentences entail other sentences. Depends on factive matrix verb hypernym vs. hyponym type of sentence:

Automatic Detection of Causal Relations for Question Answering

UNED Anselmo Peñas Álvaro Rodrigo Felisa Verdejo Thanks to…

Presentation transcript:

UNED at PASCAL RTE-2 Challenge IR&NLP Group at UNED nlp.uned.es Jesús Herrera Anselmo Peñas Álvaro Rodrigo Felisa Verdejo

Outline  Underlying thesis  Overall architecture  Lexical Entailment  Entailment decision  Results  Conclusion & future Work

Underlying Thesis Compositional meaning of the statements All “components” of the hypothesis must be entailed Definition of “statement component” Our Roadmap: 1. Word & multiword level 2. NE, numeric & temporal expressions level 3. Phrase level & paraphrase (in progress)

Overall Architecture Dependency Parser NE Recognizer Linguistic Processing WN-based Entailment NE Entailment Lexical Entailment Decision Sentence Matching Sentence Entailment

WN-based Lexical Entailment  WordNet based entailment Synonymy Hyponymy( Herrera et al. 2005) Antonymy Verb Entailment Part meronymyItaly entails Europe PertainymyItalian entails Italy (Glosses)  Search of entailment paths Depth Breadth Path Length

WN-based Lexical Entailment 10% 1% 3%

Numerical Entailment 17 million citizensMore than 15 million people TextHypothesis recognize Lower bound: 17,000,000 Upper bound: 17,000,000 Unit: citizen Lower bound: 15,000,000 Upper bound: infinite Unit: person normalize Entailment is TRUE if [17,000, ,000,000]  [15,000,000.. Infinite) and citizen entails person entailment

Numerical Entailment

Entailment Decision  Support Vector Machines % of hypothesis lemmas entailed % of hypothesis words coincidence Existence of numeric expressions not entailed in the hypothesis Existence of NE not entailed in the hypothesis (after submission)

Results Run 1Run 2 (+word coincidence) (+NE entailment) IE49.0%52.0%51.0% QA56.5%52.0%53.5% IR64.5%57.0%63.5% SUM69.0%74.5%74.0% Overall59.75%58.87%60.5%

Conclusion & Future Work  WN is nice but doesn’t help too much with an indiscriminate use (Check by source task)  Different processing for each source task  NE & numerical entailment seem promising  Pay more attention to the entailment decision Future  Phrase level entailment  Paraphrase