Identifying the tradeoffs in textual entailment: deep representation versus shallow entailment Randy Goebel Alberta Innovates Centre for Machine Learning Department of Computing Science University of Alberta Edmonton, Alberta Canada

NII Shonan Meeting Seminar 057, November 27, Fuji-san

BIRS

Science or Engineering? Google’s translation team employs no linguistics Does the challenge of textual entailment include scientific challenges? Is there an objective function/measure for “optimal” textual entailment?

NII Shonan Meeting Seminar 057, November 27, Outline A simple textual entailment framework Landmarks in the history of deep representation Building models of language entailment from annotated corpora Explanation based on multiple entailment models Summary

NII Shonan Meeting Seminar 057, November 27, Simple Textual Entailment Tom loves all women. X woman(X) loves(Tom,X) Does Tom love Sally? Loves(Tom,Sally) representation expressibility entailment precision A B

NII Shonan Meeting Seminar 057, November 27, Landmarks of deep representation … WVO Quine’s relational versus notional Russell, Hilbert, Robinson descriptive terms Montague’s formal philosophy Steedman’s induction of semantic parsing

NII Shonan Meeting Seminar 057, November 27, Quine’s Relational versus Notional

NII Shonan Meeting Seminar 057, November 27, Quine’s Relational versus Notional

NII Shonan Meeting Seminar 057, November 27, Quine’s Relational versus Notional

NII Shonan Meeting Seminar 057, November 27, Quine’s Relational versus Notional “John wants a sloop.” relationalnotional X.wants(John,X) & sloop(X) X.wants(John,X) & sloop(X) & name(X, “JohnB”)

NII Shonan Meeting Seminar 057, November 27, Russell A logic of definite descriptions

NII Shonan Meeting Seminar 057, November 27, Hilbert, Robinson A logic of indefinite descriptions

NII Shonan Meeting Seminar 057, November 27, Russell, Hilbert, Robinson “John wants a sloop.” relationalnotional

NII Shonan Meeting Seminar 057, November 27, The formal philosophy of Montague Relentless pursuit of compositionality “ In the tradition of Montague grammar... The interpretation of a language is given by a homomorphism between an algebra of syntactic representations and an algebra of semantic objects” “John wants a sloop.”

NII Shonan Meeting Seminar 057, November 27, Hall-Partee Summary of Montague

NII Shonan Meeting Seminar 057, November 27, Hall-Partee Summary “… the big difference is that Montague derives the compositionally; the semantic structure is homomorphic to the syntactic structure”

NII Shonan Meeting Seminar 057, November 27, Semantic Parsing of Steedman Inducing semantic parsers from data, e.g, induce language model fragments from abstracted representations of entailment pairs. “– Parsing directly coupled with compositional assembly of meaning representation or “logical form”; – More recently, the induction of such parsers from data consisting of string meaning pairs.”

NII Shonan Meeting Seminar 057, November 27, A privately appointed agent can be a minor, but acts of agency by a minor can be rescinded by the statutory agent. (Agent's Capacity to Act)Article 102 An agent need not to be a person with the capacity to act. A privately appointed agent can be a minor, but acts of agency by a minor can be rescinded by the statutory agent. (Agent's Capacity to Act)Article 102 An agent need not to be a person with the capacity to act. Entailment pairs as data Queries that only one team answered correctly (NII) 19

NII Shonan Meeting Seminar 057, November 27, Semantic Parsing of Steedman

NII Shonan Meeting Seminar 057, November 27, Semantic Parsing of Steedman

NII Shonan Meeting Seminar 057, November 27, Possible intermediate representations

NII Shonan Meeting Seminar 057, November 27, A spectrum of inference Correct Plausible/consistent/ambig uous Interesting Speculative Wild-ass guess Implausible Wrong deductive Non-deductive 1, 2, 3, n

NII Shonan Meeting Seminar 057, November 27, Measures of entailment quality This strategy creates elaborate “semantic parsers” which embed (fragments) of general inference processes For explanation, what properties are preserved? Soundness, completeness Are there classes of explanation processes Probability, summary, hypothesis

NII Shonan Meeting Seminar 057, November 27, John still wants a sloop “John indicated that he had been considering a variety of sail craft, and his favourite was a sloop.” “John wants a sloop.” ? 1 Sentential classifier with 83% accuracy measure 2 Semantic parsing shows “looking at a sloop” Subsumes “wanting a sloop.” 3 Model shows highest statistical correlation between these two sentences

NII Shonan Meeting Seminar 057, November 27, Summary Entailment processes need to be developed to conform to inference expectations We need annotated corporate to build semantic parsing, summary relationships, IR models, etc. by ML techniques Quality of entailment, like accuracy of inference, should be assessed by “explainability”