ISAT Working Group Augmented Cognition Eric Horvitz (Chair), Lee Kollmorgen, Randy Pausch, Paul Rosenbloom, Dylan Schmorrow, Jean Scholtz, Allen Sears, Keith Holcomb, Jack Thorpe November 2000

Challenge and Opportunity Volume of information, overall complexity of defense tasks continue to grow at a rapid pace, in stark contrast to human cognitive abilities, which remain relatively static:  Attention  Memory  Learning  Comprehension  Sensory bandwidth  Visualization abilities  Judgment and decision making  Planning and control In contrast, computational capabilities have continued to grow rapidly Apply computation to support / augment human cognition

Timing and Opportunities: Trends Human abilities Human abilities Computational resources, prowess Computational resources, prowess Augmented Cognition Today

Timing and Opportunities: Trends People People Augmented Cognition Complexity Time available Tasks Information Resources

Timing and Opportunities  Leap in computational resources, algorithms  Context sensitivity  Sensors, vision, web connectivity, multiple information sources.  Automated goal analysis  Situation awareness  Context availability  Pervasive computing, technologies supporting mobility

Augmented Cognition Efforts ExistingPsychological Results on Cognitive Limitations Target Cognitive Tasks Memory ConceptAttainment DividedAttention Computation Abilities & efficiencies  Characterizing inefficiencies, problems, opportunities re: performing critical tasks based on fundamental cognitive limitations  Identify, apply results from cognitive psychology to build computation solutions  Identify new cognitive psychology research  Identify new joint CS—cognitive psychology efforts

ExistingPsychological Results on Cognitive Limitations Target Cognitive Tasks Memory ConceptAttainment DividedAttention Abilities & efficiencies  Characterizing inefficiencies, problems, opportunities re: performing critical tasks based on fundamental cognitive limitations  Identify, apply results from cognitive psychology to build computation solutions  Identify new cognitive psychology research  Identify new joint CS—cognitive psychology efforts * New Cog. Psych. Research * New Cog. Psych. Research * HCI, Aug. Cognition Research * HCI, Aug. Cognition Research Augmented Cognition Efforts ?

Leverage Sensing, Inference about Context Level 1  Context-Sensitive UI, assistance  Situation Assessment  Respond to commands, requests in appropriate manner Computational Solutions to Basic Limitations in Memory, Judgment, Concept Acquisition Level 2  Context-sensitive reminder systems  Enhanced training tools  Reasoning about interruption, divided attention Methods for Fluid Mixed-initiative interaction Level 3  Tight, effective collaborative interaction  Natural conversation  1:n oversight of autonomous systems  Automated fielding of components of problem to user and system Rich understanding, leveraging knowledge about human cognitive capabilities and limitations Level 4  Enhanced concept acquisition  Lifelong memory agents  Judgment debiasing tools Autonomous learning, exploring, engaging users Level 5  Proactive, continual computation & exploration  Companion systems Augmented Cognition Roadmap

Some Study Directions  Foundations of Augmented Cognition Technologies (FACT)  Mixed-initiative control and monitoring within human-’bot systems  Cognitive power tools for warfighters  Smart ambient companions (SAC)  Information triage and rendering  Dynamic, scalable task organization and execution

Some Study Directions  Foundations of Augmented Cognition Technologies (FACT)  Continue to flesh out challenges and opportunities, scope terrain & directions  Case studies, as appropriate  Cognitive power tools for warfighters  Smart ambient companions (SAC)  Information triage and rendering  Mixed-initiative control and monitoring within human-’bot systems E. Horvitz would chair Seek psychologist as co-chair

Some Study Directions  Mixed-initiative control and monitoring within human-’bot systems  Tasking groups  Strategies for monitoring groups  Extending conceptual foundations of computer—human, human-human interfaces  Attentional issues  Considerations of context, content, goals  Communication under limited resources Paul Rosenbloom would chair.