SemEval && TempQuestion Linfeng Shi

SemEval The evaluations are intended to explore the nature of meaning in language, WSD, sentiment analysis, … Spatial Role Labeling start at 2012 2012, 2013 Spatial Role Labeling 2015 SpaceEval SpRL Recognize trajector, landmark, spatial indicator Predicte && classify spatial relation, E.g. There are [red umbrellas]tr [in]sp a [park]lm

SpaceEval Change annotation specification ISO-Space Elements Entities Location, spatial entity, motion, event path, path Spatial Relations Topological, relational, and distance

ISO-Space Location An inherently grounded spatial entity, such as mountains, cities, rivers, administrative entities E.g. The new tropical depression was about 430 miles (690 kilometers) west of the [southernmost Cape Verde Islandloc3], forecasters said Distance and direction as the signals Location, place, region: There location covers location and place, and is associated with a region

ISO-Space Spatial entity An entity that is not inherently a LOCATION, but one which is identified as participating in spatial relation E.g. The new [tropical depression]se was about 430 miles (690 kilometers) west of the [southernmost Cape Verde Islandloc3], forecasters said. [John] is in the car

ISO-Space Motion A motion is an inherently spatial event, involving a change of location. Motions include verbs like leave, move, approach E.g. It could become a Category 2 hurricane as [it]se [approaches]m the [Caribbean]loc

ISO-Space Path a functional location where the focus is on the potential for traversal, like road, river, Route 66, 102 Highway

ISO-Space Event Path It is defined by the motion event Difference vs Path: always associated with a motion

ISO-Space Qualitative spatial link Spatial relation between two spatial objects Topological spatial relations From RCC8 Relational spatial relations One region in terms of another “the car in front of the house” Core inventory

ISO-Space Distance Spatial signal A distance link relates two location, spatial entities and specifies the distance between them Spatial signal A relation word that locates a state or motion in region or path, provides information on the relation type of spatial relation ‘in front of’, ‘west of’, …

SpaceEval Task lists Spatial Elements identification Spatial Signal identification Motion Signal identification Motion Relation Identification (MoveLink) Qualitative Spatial Link identification Topological spatial relation Relational spatial relation

SpaceEval-datasets DCP (Degree Confluence Project) Berlitz Travel Guides from ANC (American National Corpus) Travel weblog RFC (Ride for Climate) Three kind of test datasets Only unannotated test data was provided. Manually annotated spatial elements, without attributes, were provided Manually annotated spatial elements, with attributes, were provided.

SpaceEval-Participant Baseline Logistic regression Brandeis-CRF CRF with features including Stanford POS and NER tags HRIJP-CRF-VW CRF with POS, NER, Glove word vector IXA SVM UTD

SpaceEval-Result

TempQuestions KB-QA has limitations regarding complex questions that require decomposing the input into sub- questions The need for decomposition arises for all kinds of complex question. Author focus on temporal. Definition: contains a temporal expression, a temporal signal, or whose answer is of temporal nature Four types: Explicit, implicit, ordinal, temporal answer

Temporal Expressions TimeML Four types Temporal markup language TIMEX3 Four types Date: last Thursday Time: February 5, 2018 23:59:59 CET Duration: three weeks Set: once a week Fully specified, relatively specified

Temporal Signals Temporal relation between two TimeML entities Before, during, after Ambiguities Which movies did Besson work on before his marriage... Which movie did Besson work on before his marriage… BEFORE Before and meet, ‘before’, ‘prior to’ AFTER Before_inverse and meet_inverse, ‘after’, ‘following’ OVERLAP Other relations, ‘during’, ‘while’, ‘when’

TempQuestions: Creation Source datasets Free917 (917), WebQuestions (5810), ComplexQuestions (2100) Automated detection Mining temporal expression by SUTime, HeidelTime Using temporal signals Using ordinal words by CoreNLP and dictionary Temporal answer by pattern like ‘when’, ‘what date’ Manual inspection ‘first name’, ‘Christmas carol’

TempQuestions: Analysis Average length: 8.28 Relatively verbose, increased parsing difficulty Distribution of question types Distribution of entities and relations Performance

Thx