What Is the “Context” for Contextual Vocabulary Acquisition? William J. Rapaport Department of Computer Science & Engineering Department of Philosophy Center for Cognitive Science NSF ROLE Grant REC

Outline People can figure out a meaning for a word “from context” What does “context” mean in this context?

Definition of “CVA” “ C ontextual V ocabulary A cquisition” = def the acquisition of word meanings from text –“incidental” –“deliberate” by reasoning about –contextual cues –background knowledge Including prior word-meaning hypotheses, language knowledge… without external sources of help –no dictionaries –no people

CVA: From Algorithm to Curriculum 1.Computational theory of CVA –Based on: algorithms developed by Karen Ehrlich (1995) verbal protocols (case studies) –Implemented in a semantic-network-based knowledge-representation & reasoning system SNePS (Stuart C. Shapiro & colleagues) 2.Educational curriculum to teach CVA –Based on our algorithms & protocols –To improve vocabulary & reading comprehension –Joint work with Michael Kibby Center for Literacy & Reading Instruction

People Do “Incidental” CVA We know more words than explicitly taught –Average high-school grad knows ~45K words  learned ~2.5K words/year (over 18 yrs.) –But only taught ~400/school-year ~ 4800 in 12 years of school (~ 10% of total)  Most word meanings learned from context –“incidentally” (unconsciously) How?

People Also Do “Deliberate” CVA You’re reading; You understand everything you read, until… You come across a new word Not in dictionary No one to ask So, you try to “figure out” its meaning from “context” How? –guess? derive? infer? deduce? educe? construct? predict? … –our answer: Compute it! Via inferential search of “context”/KB But what KB?

CVA as Cognitive Science Studied in: –AI / computational linguistics –Psychology –Child-language development (L1 acquisition) –L2 acquisition (e.g., ESL) –Reading education (vocabulary development) Thus far: “multi-”disciplinary Not yet: “inter-”disciplinary!

What does ‘brachet’ mean?

(From Malory’s 15 th century Morte d’Arthur [page # in brackets]) 1.There came a white hart running into the hall with a white brachet next to him, and thirty couples of black hounds came running after them. [66] People:brachet = animal? inanimate object? don’t know. Computer:brachet = physical object (because only physical objects have color) 2.As the hart went by the sideboard, the white brachet bit him. [66] People:brachet = animal Computer:brachet = animal (because only animals bite)

Malory, continued 3. The knight arose, took up the brachet and rode away with the brachet. [66] People:brachet = animal / small animal Computer:brachet = small animal (because: picked up and carried) 4. A lady came in and cried aloud to King Arthur, “Sire, the brachet is mine”. [66] People:brachet = pet / small, valuable animal Computer:brachet = small, valuable animal (because: what’s wanted is valuable)

Malory, continued 10.There was the white brachet which bayed at him fast. [72] People:brachet = dog Computer:brachet = hound (i.e., dog that hunts) (because only hounds, which are hunting dogs, bay) 18.The hart lay dead; a brachet was biting on his throat, and other hounds came behind. [86] People:brachet = hound Computer:brachet = hound (i.e., dog that hunts) (because “x and other y”  x is a y)

How (Not) to Teach CVA: Vague Strategies Clarke & Nation 1980: a “strategy” (algorithm) 1.Look at word & context; determine POS 2.Look at grammatical context E.g., “who does what to whom”? 3.Look at wider context [E.g., for clues re: causal, temporal, class-membership, etc.] 4.Guess the word; check your guess

Vague strategies: “guess the word” = “then a miracle occurs” Surely, we computer scientists can “be more explicit”!

A More Precise, Teachable Algorithm Treat “guess” as a procedure call –Fill in the details with our algorithm Convert the algorithm into a curriculum –To enhance students’ abilities to use deliberate CVA strategies

Figure out meaning of word from what? context (i.e., the text)? –Werner & Kaplan 52, McKeown 85, Schatz & Baldwin 86 context and reader’s background knowledge? –Granger 77, Sternberg 83, Hastings 94 context including background knowledge? –Nation & Coady 88, Graesser & Bower 90 Note: –“context” = text  context is external to reader’s mind Could also be spoken/visual/situative (still external) –“background knowledge”: internal to reader’s mind What is (or should be) the “context” for CVA?

Some Proposed Preliminary Definitions (to extract order out of confusion) Unknown word for a reader = def –Word or phrase that reader has never seen before –Or only has vague idea of its meaning Different levels of knowing meaning of word –Notation: “X”

Proposed preliminary definitions Text = def –(written) passage –containing X –single phrase or sentence … several paragraphs

Proposed preliminary definitions Co-text of X in some text = def –The entire text “minus” X; i.e., entire text surrounding X –E.g., if X = ‘brachet’, and text = “There came a white hart running into the hall with a white brachet next to him, and thirty couples of black hounds came running after them.” Then X’s co-text in this text = “There came a white hart running into the hall with a white ______ next to him, and thirty couples of black hounds came running after them.” –Cf. “cloze” tests in psychology But, in CVA, reader seeks meaning or definition –NOT a missing word or synonym: There’s no “correct” answer! –“Co-text” is what many mean by “context” BUT: they shouldn’t!

Proposed preliminary definitions The reader’s prior knowledge = def –the knowledge that the reader has when s/he begins to read the text –and is able to recall as needed while reading “knight picks up & carries brachet”  ? small Warnings: –“knowledge”  truth so, “prior beliefs” is better –“prior” vs. “background” vs. “world”, etc. See next slide!

Proposed preliminary definitions Possible synonyms for “prior knowledge”, each with different connotation: –Background knowledge: Can use for information that author assumes reader to have –World knowledge: General factual knowledge about things other than the text’s topic –Domain knowledge: Specialized, subject-specific knowledge about the text’s topic –Commonsense knowledge: Knowledge “everyone” has –E.g., CYC, “cultural literacy” (Hirsch) These overlap: –PK should include some CSK, might include some DK –BK might include much DK

Steps towards a Proper Definition of “Context” Step 1: –The context of X for a reader = def 1.The co-text of X 2. “+” the reader’s prior knowledge Both are needed! –After reading: “the white brachet bit the hart in the buttock” most subjects infer that brachets are (probably) animals, from: That text, plus: Available PK premise: “If x bites y, then x is (probably) an animal. –Inference is not an enthymeme! (because …)

Proper definition of “context”: But (inference not an enthymeme because): –When you read, you “internalize” the text You “bring it into” your mind Gärdenfors 1997, 1999; Jackendoff 2002 –This “internalized text” is more important than the actual words on paper: Text:“I’m going to put the cat out” Misread as:“I’m going to put the car out” –leads to different understanding of “the text” –What matters is what the reader thinks the text is, Not what the text actually is Therefore …

Proper definition of “context”: Step 2: –The context of X for a reader = def A single KB, consisting of: 1. The reader’s internalized co-text of X 2. “ + ” the reader’s prior knowledge

Proper definition of “context”: But: What is “+” ? –Not: mere conjunction or union! –Active readers make inferences while reading. From text = “a white brachet” & prior commonsense knowledge = “only physical objects have color”, reader might infer that brachets are physical objects From “The knight took up the brachet and rode away with the brachet.” & prior commonsense knowledge about size, reader might infer that brachet is small enough to be carried –Whole > Σ parts: inference from [internalized text + PK]  new info not in text or in PK I.e., you can learn from reading!

Proper definition of “context”: But: Whole < Σ parts! –Reader can learn that some prior beliefs were mistaken Or: reader can decide that text is mistaken (less likely) Reading & CVA need belief revision! operation “ + ”: –input:PK & internalized co-text –output:“belief-revised integration” of input, via: Expansion: –addition of new beliefs from ICT into PK, plus new inferences Revision: –retraction of inconsistent prior beliefs together with inferences from them Consolidation: –eliminate further inconsistencies

Prior KnowledgeText PK1 PK2 PK3 PK4

Prior KnowledgeText PK1 PK2 PK3 PK4 T1

Integrated KBText PK1 PK2 PK3 PK4 T1 I(T1) internalization

B-R Integrated KBText PK1 PK2 PK3 PK4 T1 I(T1) internalization P5 inference

B-R Integrated KBText PK1 PK2 PK3 PK4 T1 I(T1) internalization P5 inference T2 I(T2) P6

B-R Integrated KBText PK1 PK2 PK3 PK4 T1 I(T1) internalization P5 inference T2 I(T2) P6 T3 I(T3)

B-R Integrated KBText PK1 PK2 PK3 PK4 T1 I(T1) internalization P5 inference T2 I(T2) P6 T3 I(T3)

B-R Integrated KBText PK1 PK2 PK3 PK4 T1 I(T1) internalization P5 inference T2 I(T2) P6 T3 I(T3) P7 Note: All “contextual” reasoning is done in this “context”:

Proper definition of “context”: One more detail: X needs to be internalized Context is a 3-place relation among: –Reader, word, and text Final(?) def.: –Let T be a text –Let R be a reader of T –Let X be a word in T (that is unknown to R) –Let T-X be X’s co-text in T. –Then: The context that R should use to hypothesize a meaning for R’s internalization of X as it occurs in T = def –The belief-revised integration of R’s prior knowledge with R’s internalization of T-X.

This definition agrees with… Cognitive-science & reading-theoretic views of text understanding –Schank 1982, Rumelhart 1985, etc. & KRR techniques for text understanding: –Reader’s mind modeled by KB of prior knowledge Expressed in KR language (for us: SNePS) –Computational cognitive agent reads the text, “integrating” text info into its KB, and making inferences & performing belief revision along the way –When asked to define a word, Agent deductively searches this single, integrated KB for information to fill slots of a definition frame –Agent’s “context” for CVA = this single, integrated KB

Distinguishing Prior Knowledge from Integrated Co-Text So KB can be “disentangled” as needed for belief revision or to control inference: Each proposition in the single, integrated KB is marked with its “source”: –Originally from PK –Originally from text –Inferred Sources of premises

Some Open Questions Roles of spoken/visual/situative contexts Relation of CVA “context” to formal theories of context (e.g., McCarthy, Guha…) Relation of I(T) to prior-KB; e.g.: –Is I(T i ) true in prior-KB? It is “accepted pro tem”. –Is I(T) a “subcontext” of pKB or B-R KB? How to “activate” relevant prior knowledge. Etc.

Summary People can figure out a meaning for a word “from context”, where… “Context” = belief-revised integration of: –reader’s prior knowledge, with –internalized information from the text This clearer concept of relevant notion of “context” will help us: –evaluate other research –develop our curriculum