I1-[OntoSpace] Ontologies for Spatial Communication John Bateman, Kerstin Fischer, Reinhard Moratz Scott Farrar, Thora Tenbrink

I1Ontospace: Methodology and Procedures ●Overview of the ‚Ontospace‘ (KF) ●Example analysis of rechts/links ●in the linguistic ontology (KF) ○illustration of the parameters (TT) ●in the spatial ontology (SF) ●The ‚Ontospace‘ and NLP/ASP (SF)

The Problem ●How do we go from natural language to computationally meaningful, tractable representations, and back? ●Fahre zum linken Karton>[some scene desccription in computerese]

A Solution ●Hypothesis: use an intermediate semantic representation. ●The level of semantics pertains only to the conceptual distinctions the language commits to, i.e., as expressed in the grammar. ●Compare: “Fahre nach rechts." [direction] to “Fahre zum rechten Würfel.“ [attribute]

Levels of Representation ●So we have three levels of representation: ●grammatical(linguistic form) ●semantic(linguistic meaning) ●conceptual(meaning in general) ●This is represented schematically…

The ‚Ontospace‘ Conceptual Space Semantic MapGrammar

Spatial OntologyLinguistic OntologyGrammar Where do ontologies come in?

Spatial Ontology Linguistic OntologyGrammar inter ontology mediation

Spatial Ontology Linguistic OntologyGrammar computational models

Spatial OntologyLinguistic OntologyGrammar Ambiguity Parameters, e.g. reference system chosen, perspective taken

Spatial Ontology Linguistic OntologyGrammar situational parameterization

Spatial Ontology Linguistic OntologyGrammar situational parameterization experiments

linguistic behaviour concept of robot concept of language concept of space concept of situation

linguistic behaviour concept of robot concept of language concept of space concept of situation choice of terms/ constructions discourse structure, politeness, prosody, instructional strategies, perspective taking, discourse marking address forms, alignment, etc.

linguistic behaviour concept of robot concept of language concept of space concept of situation

linguistic behaviour concept of robot concept of language concept of space concept of situation pre conceptions appearance robot behaviour

linguistic behaviour concept of robot concept of language concept of space concept of situation task orientation as experiment understanding at issue

linguistic behaviour concept of robot concept of language concept of space concept of situation Negotiation discourse history alignment success/ failure shaping

Experimental elicitation of human-robot interaction ●Aims: ●to identify the linguistic structures used in different situations ●to identify the parameters that trigger the use of the different linguistic structures possible, i.e. understand the conditions of their usage ●in order to: ●adapt the ASP/NLP system components to the actually occurring (e.g. the lexicon/word list (limitations of current speech recognizers), the parser, the semantic/pragmatic representations, the discourse manager) ●respond appropriately to the users' linguistic behaviour (generation, discourse strategies, shaping)

Experimental results regarding rechts/links ●Types of grammatical constructions: pos=part of speech LO=linguistic ontology (based on GUM) zum rechten Würfel pos= adjective LO= relative property

Experimental results regarding rechts/links der Würfel rechts des Kartons pos= preposition LO= direction rechts halten pos=> noun LO=> bounded spatial region

Experimental results regarding rechts/links fahre nach rechts pos=> noun LO=> unbounded region fahre rechts pos=> adverb LO=> direction

Experimental results regarding rechts/links zum Würfel rechts pos=> adjective (syntactically not integrated) LO=> relative property rechts vor dem Würfel pos=> preposition (part of complex) LO=> direction

Experimental results regarding rechts/links zweite von rechts pos=> noun LO=> space point die Tasse rechts vor dir pos=> preposition (part of complex) LO=> direction

Experimental results regarding rechts/links drehe Dich nach rechts pos=> noun LO=> direction rechts von Dir pos=> adverb LO=> direction

Situational variables influencing users’ choice of terms ●Experiment 1 (N=541): ●HRI: ○pioneer ○robot understands only goalbased utterances ○one task at a time ○delay because of real processing ●rechts/links: 44% ●rechte/linke: 55% ●Experiment 2 (N=137): ●Woz scenario: ○aibo ○fixed behaviour ○one task at a time & sequence of Ols ○no delay ●rechts/links: 88% ●rechte/linke: 11%

Situational differences trigger different uses of links/rechts Exp. 1: “Gehe zum linken Karton” Exp. 2: “links”

Ambiguity of “links” ●without context: 90°angle ●with road crossing or curve: street’s angle ●with object localisation: towards object in region

Situational parameters influence the linguistic behaviour observable ●“Gehe zur linken Kiste” ●in general: contrastive use of adjectives (depending on the presence of competing objects ) ●here: discourse history biases towards this usage: users had previously experienced success with this syntactic format

Where are we? “Gehe zur linken Kiste”

Available Tools ●Spatial ontology toolkit? ●SUMO, DOLCE, BFO ●OpenCyc ●Linguistic ontology toolkit? ●Generalized Upper Model

Steps for Constructing a “Scene” ●Identify the environment ●Identify objects in environment ●Identify relationships among various objects

Configuration in OpenCyc ●“A Configuration is a static situation consisting of two or more PartiallyTangible things of certain types standing in a certain type of spatial relationship (or set of relationships). ”

Configuration in the Taxonomy Configuration StaticSituation SpatialThingLocalized SpatialThing Thing Situation … …

Objects in the Configuration

The Objects SpatialThing ToAndBottom SidedObject Obj1, Obj2 LeftAndRight SidedObject HexalateralObject VertabrateLandTransportationDevice HumUser Pioneer …

So far so good… ●(instance Obj1 SymmetricalObject) ●(instance Obj2 SymmetricalObject) ●(instance HumUser Human) ●(instance Pioneer Robot)

Spatial Relationships in the Configuration

Spatial Relations in OpenCyc parallelObjects notFarFrom connectedTo spatiallyDisjoint physicalPartsdisjoint onSamePlanetSurfaceAs spatiallyRelated fitsIn perpendicularObjects pointingTowards aligned connectedTo parallelObjects

Zooming in… notFarFrom inRegion near spatially Contains spatially Includes stuckTo inFrontOf touches wears Clothing

inFrontOf… ●“(inFrontOfGenerally FORE AFT) means that FORE is in front of the tangible object AFT. More precisely, the intrinsic backtofront axis of AFT is within 45 degrees of some line intersecting both FORE and AFT.” ●“(inFrontOfDirectly FORE AFT) means that FORE is directly in front of tangible object AFT. More precisely, it implies both (inFrontOfGenerally FORE AFT) and that there is at least one line parallel to the forward pointing axis of AFT that intersects both FORE and AFT.”

Intrinsic Bias ●Maybe this is okay for the level of spatial representation.

Relations ●(behindDirectly Obj2 Pioneer) ●(inFrontOfDirectly Pioneer Human)

Relations ●How do we represent that Obj1 is to the left of Pioneer?

Left/Right in OpenCyc ●OpenCyc has no leftOf/rightOf relations. ●How can these relative relations be defined using the tools of OpenCyc? ●Either: ●Fall back on some absolute direction system (North, West, etc.) ●Locate intrinsic object parts (LeftSide, FrontSide, etc.)

WordNet: An Aside ●WordNet uses the first option: ●left (location near or direction toward the left side; i.e. the side to the north when a person or object faces east; "she stood on the left") ● => stage left, left stage (the part of the stage on the actor's left as the actor faces the audience)

Back to OpenCyc ●Recall that OpenCyc provides the taxonomic framework for intrinsic object parts.

The Objects SpatialThing ToAndBottom SidedObject Obj1, Obj2 LeftAndRight SidedObject HexalateralObject VertabrateLandTransportationDevice HumUser Pioneer …

LeftAndRightSidedObject “A LeftAndRightSidedObject is an object with an intrinsic left side and an intrinsic right side. By 'intrinsic' we mean simply that there exists an established convention according to which one side is considered 'Left' and the other side is considered 'Right'. Usually, the two sides can be reliably distinguished from one another. ”

Identifying the Left Side ●Use the handy LeftRegionFn: “The function (LeftRegionFn REGOROBJ), applied to a region or object REGOROBJ, means the region consisting of the left half or flank or left main portion of REGOROBJ. It applies only when REGOROBJ itself has an intrinsic left/right orientation, or is part of a larger region or object that has a left/right orientation.”

Relations ●(leftOf Obj1 Pioneer) ●(rightOf Human Pioneer) ●(leftOf Pioneer Human) …

Left/Right in OpenCyc ●This assumes a static configuration viewed from Pioneer’s perspective ●What if the perspective changes during the dialog?

speech recognition agents speech recognition output interface where is the beer? analysis component (CG based: string  semantic representation) dialog m a n a g e r generation component speech synthesis agents string with speech synthesis markup in the fridge! Architectural overview (I1/I3)

Relation of I1-[OntoSpace] to other projects using ontologies

Summary: Ontospace working group ●Presented our methodology ●From grammatical description ●To linguistic semantic representation (linguistic ontology) ●And to conceptual representation (spatial ontology)

Summary: Ontospace working group ●Presented first results regarding links/rechts ●Presented first explorations regarding parameterization ●Presented available spatial tools from a large-scale conceptual ontology (OpenCyc)