1. SemEval && TempQuestion
Linfeng Shi

2. SemEval
The evaluations are intended to explore the nature of meaning in language, WSD, sentiment analysis, …
Spatial Role Labeling start at 2012
2012, 2013 Spatial Role Labeling
2015 SpaceEval
SpRL
Recognize trajector, landmark, spatial indicator
Predicte && classify spatial relation,
E.g. There are [red umbrellas]tr [in]sp a [park]lm

3. SpaceEval Change annotation specification ISO-Space Elements Entities
Location, spatial entity, motion, event path, path
Spatial Relations
Topological, relational, and distance

4. ISO-Space
Location
An inherently grounded spatial entity, such as mountains, cities, rivers, administrative entities
E.g.
The new tropical depression was about 430 miles (690 kilometers) west of the [southernmost Cape Verde Islandloc3], forecasters said
Distance and direction as the signals
Location, place, region:
There location covers location and place, and is associated with a region

5. ISO-Space Spatial entity
An entity that is not inherently a LOCATION, but one which is identified as participating in spatial relation
E.g.
The new [tropical depression]se was about 430 miles (690 kilometers) west of the [southernmost Cape Verde Islandloc3], forecasters said.
[John] is in the car

6. ISO-Space
Motion
A motion is an inherently spatial event, involving a change of location.
Motions include verbs like leave, move, approach
E.g.
It could become a Category 2 hurricane as [it]se [approaches]m the [Caribbean]loc

7. ISO-Space
Path
a functional location where the focus is on the potential for traversal, like road, river, Route 66, 102 Highway

8. ISO-Space Event Path It is defined by the motion event
Difference vs Path: always associated with a motion

9. ISO-Space Qualitative spatial link
Spatial relation between two spatial objects
Topological spatial relations
From RCC8
Relational spatial relations
One region in terms of another
“the car in front of the house”
Core inventory

10. ISO-Space Distance Spatial signal
A distance link relates two location, spatial entities and specifies the distance between them
Spatial signal
A relation word that locates a state or motion in region or path, provides information on the relation type of spatial relation
‘in front of’, ‘west of’, …

11. SpaceEval Task lists Spatial Elements identification
Spatial Signal identification
Motion Signal identification
Motion Relation Identification (MoveLink)
Qualitative Spatial Link identification
Topological spatial relation
Relational spatial relation

12. SpaceEval-datasets DCP (Degree Confluence Project)
Berlitz Travel Guides from ANC (American National Corpus)
Travel weblog RFC (Ride for Climate)
Three kind of test datasets
Only unannotated test data was provided.
Manually annotated spatial elements, without attributes, were provided
Manually annotated spatial elements, with attributes, were provided.

13. SpaceEval-Participant
Baseline
Logistic regression
Brandeis-CRF
CRF with features including Stanford POS and NER tags
HRIJP-CRF-VW
CRF with POS, NER, Glove word vector
IXA
SVM
UTD

14. SpaceEval-Result

15. TempQuestions
KB-QA has limitations regarding complex questions that require decomposing the input into sub- questions
The need for decomposition arises for all kinds of complex question. Author focus on temporal.
Definition: contains a temporal expression, a temporal signal, or whose answer is of temporal nature
Four types:
Explicit, implicit, ordinal, temporal answer

16. Temporal Expressions TimeML Four types
Temporal markup language
TIMEX3
Four types
Date: last Thursday
Time: February 5, :59:59 CET
Duration: three weeks
Set: once a week
Fully specified, relatively specified

17. Temporal Signals Temporal relation between two TimeML entities
Before, during, after
Ambiguities
Which movies did Besson work on before his marriage...
Which movie did Besson work on before his marriage…
BEFORE
Before and meet, ‘before’, ‘prior to’
AFTER
Before_inverse and meet_inverse, ‘after’, ‘following’
OVERLAP
Other relations, ‘during’, ‘while’, ‘when’

18. TempQuestions: Creation
Source datasets
Free917 (917), WebQuestions (5810), ComplexQuestions (2100)
Automated detection
Mining temporal expression by SUTime, HeidelTime
Using temporal signals
Using ordinal words by CoreNLP and dictionary
Temporal answer by pattern like ‘when’, ‘what date’
Manual inspection
‘first name’, ‘Christmas carol’

19. TempQuestions: Analysis
Average length: 8.28
Relatively verbose, increased parsing difficulty
Distribution of question types
Distribution of entities and relations
Performance

20. Thx