The Internal World Models Needed to Perform Situation Estimation Jim Eilbert AP Technology 6 Forrest Central Dr, Titusville, NJ

Modeling a Phenomenon that Exercises a Large Portion of the Cognitive Machinery Consider cognitive processes relevant for a robot with sensory-driven behavior Basic requirement is a feedback loop, e g. OODA Loop widely used model of military decision making 2 Observe ActDecide Orient World Expectation

Expanded OODA Loop Needed to Deal with Predictive Errors and Sensory Uncertainty 3 Hypothesize Alternate Situations Uncertain estimate Select Collection Action Appraise Collection Plans Plan Information Collection to Reduce Uncertainty Perception of Objects & Actions Act Continue High- Level Behavior Situation Update Collision avoidance Appraise Future Situations vs. Motives & Goals Select Action Situation Estimation Project Feasible Behaviors Unexpected object or action Recognize termination condition Appraise Estimated Situation

Drivers for Metareasoning Multiple competing objectives – What is the relative importance of the different objectives Predictions about the effect of running a behavior are untrustworthy – Are deviations from expectations too large to continue? Isolated perceptions are notoriously untrustworthy – Is current estimate good enough to act on? 4 Act Goal Attainment Collision Avoidance Uncertainty Reduction Situation Estimation Novel or Unpredicted Event Active Collection Uncertain Situation

Internal Models Used in the OODA Process People store various types of relational information including -- Spatial, Functional, Episodic, Categories Sensor Coordinates Functional & Spatial Models Functional & Spatial Models Fill in missing information Object /Event Recognition “Global” Coordinates A reasoning frame containing the objects of current importance is built on-the-fly Episodic or High-level Behavior Models Episodic or High-level Behavior Models Run Behaviors in Reasoning Frame Project Feasible Behavior

Reasoning Frames for Doing Situation Estimation, Appraisal, & Action Selection “Mental spaces are very partial assemblies constructed as we think and talk, for purposes of local understanding and action.” (Fauconnier, Turner 2002). 6 PROPOSITION A Mental Space is a region within the vast web of stored relational data that contains the actors of current interest and the information needed to run and appraise their behaviors

Roseman’s Appraisal Model of Emotion Five secondary dimensions – Unexpectedness of the event – Agency: What or who caused the motive-relevant event? – Problem type: Is problem specific to the circumstance or intrinsic? – Control potential: Can something be done about an expected event? – Probability of the motive relevant event: Is outcome uncertain or definite? 7 Getting what you wantGetting what you don’t want Not getting what you wantNot getting what you don’t want Motivational State Situational State

Emotions and Mental Space Switching Emotion could supply the jolt needed to overcome Mental Space hysteresis and cause a reset EXAMPLES Unexpectedness – Surprise  discrepancy between expectations & sensed information – Current Mental Space is switched to a new Mental Space constructed around salient features causing the surprise. Agent’s actions can cause a negative motive situation leading to anger at the agent the agent’s character – The bad action can lead to a revised model of the agent’s character – Anger can cause a switch to a new Mental Space with the revised model of the agent 8 Mental Space Switching is a new role for emotions. Previously suggested roles -- social communication and learning

9

Storing Complex Relational Information for Recognition and Behavior Projection Spatial memory or internal maps of 2D and 3D structure or (Allen 2004) Functional groupings, i.e. things that can be used for the same task – Saws, axes, and logs are all associated with getting firewood (Luria 1974) Episodic memory (Tulving 2001) captures causal sequences – Includes what, where and when information Ontologies and category hierarchies – Saws, axes, screwdrivers and can openers are all tools, although they are unlikely to be used on the same task 10 Relational information forms a vast, tightly coupled web

Inferring Information that Was Not Sensed Directly from Relational Information Each type of relational information allows different types of information to be inferred – Spatial memory  presence and location of unseen objects (Nissanov, et.al. 2002) – Functional groups  activity in a video sequence from the presence of objects in the same group (2007) – Narrative sequences  predict future events based on the occurrence of part a sequence (Eilbert, et.al. 2002) – Ontologies are used to determine legal substitutions in any of the othe types of relational information Cat’s make inferences using spatial memory and functions groups, and may be able limited causal structure using their episodic-like memory 11

Evolutionary Drivers for Sophisticated REASONING Reasoning Drivers Many Behavioral Choices Task complexity, i.e. number of decisions Diversity of the world 12 Reasoning is needed to select among behaviors Simple predators (say frogs) perceive a world containing – Large and small moving objects – Obstacles and open pathways Tool users must perceive a much richer world Reasoning about how to do a task can be arbitrarily hard

Evolutionary Roots of Metacognition Animal metacognition? – A cat is stalking a mouse is oblivious to other objects Has it decided that the risk of losing the mouse outweighs the safety risk in turning off its normal monitoring? – Why don’t cats bother chasing squirrels into trees? Is there metareasoning going on that tells the cat that it is now in an environment that gives a big advantage to the squirrel, or Is giving up just a conditioned response? Experimental Evidence – Cats have internal world models: Spatial memory, independent of the current viewing direction, is also seen in reptiles and birds (Gallistel 1990) Episodic-like memory that allows ‘when’ information, as well as ‘what’ and ‘where’ to be remembered (Clayton, et.al. 1999) Dere, et.al. (2006) has found evidence of'metacognition', 'conscious recollection' of past events, and 'temporal order memory‘ in animals – The learning curve during conditioning suggests that once a mammal “gets it,” they complete learning in just a few trials (Gallistel 2004). 13