Topics Beyond the imitation game Reading the web Turing test Winograd schema challenge Reading the web NELL

TURING Alan Turing is the father of both theoretical computer science and of AI He was the first to imagine machines able of intelligent behavior Devised an intelligence test, the TURING TEST: replace the question “Can a machine be endowed with intelligence” with the question: Can a machine display such human-like behavior to convince a human observer that it is a human being?

THE TURING TEST Alan M. Turing, “Computing Machinery and Intelligence”, 1950 (in Boden, Philosophy of AI) The question: what does it mean for a machine to be intelligent?

Turing Test

Eugene Goostman On June 7, 2014, a computer program named Eugene Goostman was declared to be the first AI to have passed the Turing Test in a competition held by the University of Reading in England. In the competition Eugene was able to convince 33% of judges that they were talking with a 13-year-old Ukrainian boy.[6]  Critics claimed that Eugene passed the test simply by fooling the judge and taking advantages of its purported identity. For example, it could easily skip some key questions by joking around and changing subjects. However, the judge would forgive its mistakes because Eugene identified as a teenager who spoke English as his second language.[7]

Weaknesses of the Turing Test The event of Eugene Goostman exhibited some of the problems of the Turing Test. Levesque[2]identifies three major issues: Deception: The machine is forced to construct a false identity, which is not part of intelligence. Conversation: A lot of interaction may qualify as "legitimate conversation"—jokes, clever asides, points of order—without requiring intelligent reasoning. Evaluation: Humans make mistakes and judges often would disagree on the results.

DARTMOUTH In 1956 a group of researchers including J. McCarthy, M. Minsky, C. Shannon, N. Rochester organized a workshop at Dartmouth to study the possibility of developing machine intelligence

The Winograd Schema Challenge Hector Levesque Commonsense 2011 Winograd schema: A pair of sentences, differing in one or two words, with an ambiguity that is resolved oppositely.

Winograd Schema Challenge an alternative to the Turing Test that provides a more accurate measure of genuine machine intelligence Rather than base the test on the sort of short free-form conversation suggested by the Turing Test the Winograd Schema Challenge poses a set of multiple-choice questions that have a form where the answers are expected to be fairly obvious to a layperson, but ambiguous for a machine without human-like reasoning or intelligence.

COMMONSENSE KNOWLEDGE IN LANGUAGE UNDERSTANDING Winograd (1974): The city council refused the women a permit because they feared violence. The city council refused the women a permit because they advocated violence

The trophy would not fit in the brown suitcase because it was too big The trophy would not fit in the brown suitcase because it was too big. What was too big? The trophy would not fit in the brown suitcase because it was too small. What was too small?

Joan made sure to thank Susan for all the help she had given Joan made sure to thank Susan for all the help she had given. Who had given the help? Joan made sure to thank Susan for all the help she had received. Who had received the help?

Winograd Schema Challenge Collecting corpus of schemas that are effortlessly disambiguated by human readers sound natural. can’t be solved using selectional restrictions are not easily Googlable. Challenge for AI. Less far-reaching than the Turing Test, but less problematic.

Winograd Schema Challenge provides us with a tool for concretely measuring research progress in commonsense reasoning, an essential element of our intelligent systems. 

Advantages The Winograd Schema Challenge has the following purported advantages: Knowledge and commonsense reasoning are required to solve them. Winograd Schemas of varying difficulty may be designed, involving anything from simple cause-and-effect relationships to complex narratives of events. They may be constructed to test reasoning ability in specific domains (e.g., social/psychological or spatial reasoning). There is no need for human judges.[4]

Various ‘reference resolution’ levels in the Winograd Schema

Examples The Syntactic/Data Level The Semantic/Information Level This is the level at which only one of the two noun phrases is the correct referent and where simple syntactic data is enough to resolve the reference. Here are some typical examples:(6)   John  informed Mary  that he passed the exam.  (reference here can be easily resolved by gender data) The Semantic/Information Level semantic information (usually type information that might be available in a strongly-typed ontology) are enough to resolve the reference. Here is a typical example:(7) Our graduate students  published 20 papers  this year and, apparently, few of them also authored books