LING 581: Advanced Computational Linguistics Lecture Notes February 16th

Administrivia Homework presentations today A related experiment … New topic: computational semantics

Homework – WSJ corpus: sections 00 through 24 – Training: normally (20 sections) concatenate mrg files as input to Bikel Collins train training is fast – Evaluation: on section 23 use tsurgeon to get data ready for EVALB use Bikel Collins parse repeatedly on wsj-23.lsp sentences, parsing is slow – Question: How does the Bikel Collins vary in precision and recall? if you randomly pick 1, 2, 3 up to 20 sections to do the training with… – Use EVALB plot precision and recall graphs

Perl Code Generate training data

Training Data

An experiment

Parsing Data All 5! (=120) permutations of – colorless green ideas sleep furiously.

Parsing Data The winning sentence was: ①Furiously ideas sleep colorless green. after training on sections (approx. 40,000 sentences) sleep takes ADJP object with two heads adverb (RB) furiously modifies noun sleep takes ADJP object with two heads adverb (RB) furiously modifies noun

Parsing Data The next two highest scoring permutations were: ②Furiously green ideas sleep colorless. ③Green ideas sleep furiously colorless. sleep takes NP objectsleep takes ADJP object

Parsing Data (Pereira 2002) compared Chomsky’s original minimal pair: 23.colorless green ideas sleep furiously 36.furiously sleep ideas green colorless Ranked #23 and #36 respectively out of 120

Parsing Data But … graph (next slide) shows how arbitrary these rankings are: – furiously sleep ideas green colorless Vastly outranks – colorless green ideas sleep furiously (and the top 3 sentences) when trained on randomly chosen sections covering 14,188 to 31,616 WSJ PTB sentences

Rank of Best Parse for Sentence Permutation Sentence

Rank of Best Parse for Sentence Permutation Note also that: 23.colorless green ideas sleep furiously totally outranks the top 3 (and #36) just briefly on one data point (when trained on sentences)

New Topic Computational Semantics (Portner 2005), an intro book titled What is meaning?, asks on page 1 – Why formalize? can construct precise theories precise theories are better – “they don’t allow the theorists to fudge the data quite so easily as less precise theories do” – Why implement?

Formalization Challenge: – we all understand the argument made on the previous page? – How do we formalize it? – Why formalize? can construct precise theories precise theories are better – “they don’t allow the theorists to fudge the data quite so easily as less precise theories do”

Implementation We can implement in Prolog, see next time … (Make sure you have SWI Prolog up and running for next time) – free download from

Simple past vs. present perfect – (1) Mary received the most votes in the election – (2) Mary has received the most votes in the election – (so) Mary will be the next president Idea – (1) reports a past event – (2) reports a past event and a current “result” – present perfect expectation... (entailment)

Simple past vs. present perfect – (3) Will Mary be able to finish Dos Passos’ USA trilogy by the next club meeting? It’s so long! – Well, she has read Remembrance of Things Past, and its even longer – what’s a “result” here? expectation... (entailment) Mary has read a really long book before and therefore...

Meaning What is a Meaning? – difficult sometimes to pin down precisely – by reference to other words – foreign language: 犬 (inu) = “dog”

Meaning What is a Meaning? Example: – important – Merriam-Webster (sense 1): marked by or indicative of significant worth or consequence : valuable in content or relationship

Question What is a Meaning? Example: – important – Thesaurus – Text: 1 having great meaning or lasting effect – – Synonyms big, consequential, eventful, major, material, meaningful, momentous, significant, substantial, weighty – Related Words decisive, fatal, fateful, strategic; earnest, grave, serious, sincere; distinctive, exceptional, impressive, outstanding, prominent, remarkable; valuable, worthwhile, worthy; distinguished, eminent, great, illustrious, preeminent, prestigious; famous, notorious, renowned; all- important, critical, crucial

Question What is a Meaning? Meaning = Concept (or thought or idea) – “dog” maps to DOG – maps to Problems – need to provide a concept for every meaningful piece of language – how about expressions “whatever”, “three” – need to map different expressions into same concept – Externalization? Putnam’s twin earth experiment (H 2 O vs. XYZ) – DOG a shared concept?

Question What is a Meaning? Meaning = Concept (or thought or idea) – twin earth experiment – same except H2O = XYZ – “water” refers to H2O – “water” refers to XYZ – identical twins on the two earths don’t mean the same thing by the word “water”

Question What is a Meaning? Meaning = Concept (or thought or idea) Skip the “Meaning = Concept” definition reason the word “dog” means the same thing for you and me – not that we have the same mental constructs relating to the word – it’s because of our intention to apply the word “dog” to the same things out there in our environment

Truth Conditions – The circle is inside the square Can draw a picture of scenarios for which the statement is true and the statement is false truth-conditions different from truth-value

Truth Conditions – The circle is inside the square Proposition expressed by a sentence is its truth-conditions i.e. sets of possible worlds (aka situations)

Truth Conditions – The circle is inside the square and the circle is dark – and = set intersection – Mary is a student and a baseball fan

Truth Conditions – Mary and John bought a book – and = set intersection ? are Mary and John sets? how about “and = set union”?

Truth Conditions – The square is bigger than the circle – The circle is smaller than the square – Given these two sentences, evaluate Synonymous – (what does this mean wrt truth conditions?) Contrary Entailment Tautology

Practice 1. Does sleep entail snore? 2. Does snore presuppose sleep? 3. Given the statement “All crows are black”, give an example of a sentence expressing a tautology involving this statement?