From Semantic Similarity to Semantic Relations Georgeta Bordea, November 25 Based on a talk by Alessandro Lenci titled “Will DS ever become Semantic?”, Jan 2014

 Distributional Semantics (DS)  Current Challenges in DS  Semantic Relations (Hypernymy) in DS  Directional Similarity Measures  Evaluation and Results  Conclusions Outline 1

 From contexts … Distributional Semantics 2

 … to Distributional Vectors Distributional Semantics 3

 Measuring vector similarity Distributional Semantics 4

 Semantic similarity  synonymy, categorization, etc.  Selectional preferences  semantic typing, co-composition, etc.  Context-based semantic phenomena  sense-shifts, gradience, world knowledge integration, etc.  Figurative language  analogy, metaphor, etc.  Cognitive modeling  semantic priming, similarity judgements, thematic fit, etc. Success stories 5

 Function words  negation, quantification, logical connectives, discourse particles, etc.  Intensionality  tense, aspect, modality, etc.  Reference and coreference  indexicals, anaphora, etc. Limitations of Distributional Semantics 6

 Polysemy  sense induction, regular polysemy, etc.  Compositionality  adjectival modification, predicate- argument structures, etc.  Semantic relations  hypernymy, antonymy, etc.  Inference  lexical entailments, presuppositions, implicatures, etc. Current Challenges 7

 Paradigmatic semantic relations (Lyons 1977, Cruse 1986, Fellbaum 1998, Murphy 2003) Synonymy: sofa - couch Hyperonymy: dog - animal Co-hyponymy: dog - cat Antonymy: dead - alive Meronymy: wheel - car Semantic Relations 8

 Similar words differ for the type of relation holding between them  dog is similar to animal and cat, but animal is an hypernym and cat is a coordinate (co-hyponym)  DSMs  quantitative correlate of semantic similarity (relatedness)  no discrimination between different types of semantic relations  cf. WordNet instead provides a “typed” semantic space Semantic Similarity vs. Semantic Relations 9

dog (window size = 2) cat0.77 horse0.67 fox0.65 pet0.63 rabbit0.61 pig0.57 animal0.57 mongrel0.56 sheep0.55 pigeon0.54 deer0.53 rat0.53 bird0.53 Distributional Neighbours from BNC 10 good (window size = 2) bad0.68 excellent0.66 superb0.48 poor0.45 improved0.43 improve0.43 perfect0.42 clever0.42 terrific0.42 lucky0.41 smashing0.41 improving0.41 wonderful0.41

 Pattern-based approaches (Hearst 1992)  Hypernymy “is a kind of”, “such as”  Antonymy “but not”, “or”  Analogy-based approaches (Turney 2006, 2012, 2013; Baroni and Lenci 2010, Mikolov et al. 2013)  Hypernymy dog:animal = car:vehicle Other Approaches for Extracting Semantic Relations 11

 Hypernymy is an asymmetric relation X is a dog implies X is an animal X is an animal does not imply X is a dog  Hypernyms are semantically broader terms than their hyponyms extensionally broader animal refers to a broader set of entities than dog intensionally broader animal has more general properties than dog (e.g. bark) superordinates are less informative than basic level concepts (Murphy 2002)  Distributional Inclusion Hypothesis (DIH) (Kotlerman et al. 2010) Hypernymy in Distributional Semantics 12

Directional Similarity Measures 13

 26,554 tuples expressing a relation between a target concept and a relatum concept  200 basic-level nominal concrete concepts  8 relation types (instantiated by N, V, J)  Resources - WordNet, ConceptNet, Wikipedia, etc. Gold Standard Dataset - BLESS (Baroni and Lenci 20011) 14

 Distributional features are syntactically typed collocates: subj intr-sing, obj-read, subj tr-read, etc.  Context weighting function Positive Local Mutual Information (LMI)  The Distributional Memory corpus billion tokens resulting from concatenating  ukWac (1.915 billion tokens, Web-derived texts)  English Wikipedia (820 million tokens, mid- 2009)  British National Corpus (95 million tokens) tokenized, POS-tagged, lemmatized with TreeTagger, dependency-parsed with the MaltParser Distributional Memory Corpus (Baroni and Lenci 2010) 15

 For each relation R, AP is computed for each of the 200 BLESS target concepts Evaluation with Average Precision (AP) 16

 Mean AP values for each semantic relation Results 17

 Identifying semantic relations using DS is a promising research direction  Directional similarity measures have applications in taxonomy construction  Ongoing work: evaluate and improve our global generality measure Conclusion 18