Collaborative Time-Sensitive Targeting Mary (Missy) Cummings, Stacey Scott Humans and Automation Laboratory {missyc,

Research in the Humans and Automation Lab (HAL) focuses on the multifaceted interactions of human and computer decision-making in complex sociotechnical systems. Time-Sensitive Operations for Distributed Teams Measurement of Display Complexity Collaborative Human Computer Decision Making Integrated Sensor Decision Support Reduced Manning for Unmanned Vehicle Teams Single Operator Control of Multiple Unmanned Aerial Vehicles Current HAL Research Projects

Human Supervisory Control in NCO

Supporting Supervisors of Teams of HSC Operators Supporting the Control of Teams of Heterogeneous Unmanned Vehicles Supervisory Control & Teams

How do operators manage multiple supervisory control processes? What effects do human performance limitations have on the overall system? Reduced Manning Combining Shadow positions Multiple UV control for a single operator Distributed control Multi-Modal Watch Station Inverting Current UAV Control Schemes

Tomahawk Strike Planning: missions and missiles Determine how humans and computer optimization algorithms can complement each other Cognitive strategies Collaborative Decision Making

Decision Support for Planetary Exploration Astronaut traversal replanning task Path replanning with increasingly complex constraints Affect of level of automated decision support Aiding Human-Computer Path Planning

Goal of this Research Project To identify or develop technologies to facilitate collaborative decision-making in the time-sensitive targeting task domain

Time-Sensitive Targeting is Highly Collaborative

Teamwork in Network-Centric Operations Highly connected operators, often working remotely Heavy reliance on commercially available collaboration technologies - -instant messaging (‘chat’) -video & desktop conferencing

Benefits of Collaboration Technologies Facilitate collaboration with remote colleagues Facilitate sharing of and access to digital media during group work (in both co-located and distributed settings) Provide opportunity and support for casual interactions Promote ad hoc collaboration & communication Enhance team cohesion

Disadvantages of Collaboration Technologies Require explicit effort to access / share information -implicit or more subtle communication / interaction is often missing -lack of non-verbal cues Typically convey information about completed activities, ongoing progress/status information is also important: -Has anyone seen my request for help? -Is someone working on the problem? What’s their status? Associated costs with using collaboration technologies: -distraction -time & mental effort of asking for or sharing information

Enhancing Teamwork in Non-Military Domains Research in the CSCW domain has been focused on providing technological support for teamwork in many collaborative settings: -corporate, educational, research Reviews of the CSCW literature has revealed common threads across many new approaches to CTs: -Minimizing overhead costs of sharing information about remote collaborators -Increased use of automation to help provide this information Many approaches are aimed at supporting the shared activity process

Supporting the Shared Activity Process Focused on providing team member’s activity awareness (Carroll et al., 2003): -awareness of project work that supports group performance in complex tasks -implies an awareness of other people’s plans and understandings IBM’s Activity Explorer (Millen et al., 2005) Project View IM (Scupelli et al., 2005) Virtual School Project Timeline (Carroll et al., 2003)

Supporting the Shared Activity Process (cont’d) Approaches for supporting activity awareness: Feedthrough of action Visibility of action Scupelli, P., Kiesler, S., Fussell, S.R., & Chen, C. (2005). Project View IM: A Tool for Juggling Multiple Projects and Teams. In Extended Abstracts of CHI 2005: ACM Conference on Human Factors in Computing Systems, pp Project View IM (Scupelli et al., 2005)

Supporting the Shared Activity Process (cont’d) Approaches for supporting activity awareness: Feedthrough of action Visibility of action Important to situate this information within the context of the shared activity process Virtual School Project Timeline (Carroll et al., 2003) Carroll, J.M., Neale, D.C., Isenhour, P.L., Rosson, M.B. & McCrickard, D.S. (2003). Notification and Awareness: Synchronizing task-oriented collaborative activity. International Journal of Human-Computer Studies, 58(5), pp

Challenges Introduced by the TST Domain Determining what constitutes relevant activities when ‘shared objects’ are not necessarily ‘files’ Reducing the cost of providing activity-related information to team members Balancing awareness and information overload Balancing individual performance and team performance

Methodology for Developing Activity Awareness Support for Collaborative TST Operations Develop representative collaborative TST task scenario Conduct task analysis on scenario to develop collaborative technology requirements, focusing on the shared activity process Develop initial team display designs for collaborative TST scenario incorporating CT requirements Implement team display designs Develop evaluation protocol and teamwork metrics for evaluation of team displays Design and build team testing environment Evaluate team displays Iterate team display designs based on study results

Generalized Team TST (GTT) Task A convoy is scheduled to pass through a hostile region Team of UAV operators must protect convoy until convoy exits region Stakeholders: -Convoy -UAV Operations Team + UAVs -Strike Team -Intelligence -Enemy Representative Task Scenario: GTT Task

convoy Enemy (TSTs) intel controls TST BDA target pinpoint UAVs 1A2A3A 1B 1C 1D 2B 2C 2D 3B 3C 3D ISR detects location TST ISR engages TST alert orders engagement BDA convoy location requests BDA strike team report to upper command UAV Operations Team detects TST ISR TST BDA Activity Awareness Information Analysis for GTT Task Scenario: Inter-team Interaction

Activity Awareness Information Analysis for GTT Task Scenario: Intra-team Interaction UAV Operations Team controls TST BDA (UAVs) TST ISR (intel) TST alerts (convoy) orders engagement BDA (strike team) convoy location (intel) requests BDA report to command TST ISR (UAVs) TST BDA (intel) authorize engagement target allocation mCDR request engagement BDA assignments UAV Ops situation update COM ISR – STR – BDA report – control

Activity Awareness Requirements for the GTT Task Scenario Types of intra-team shared activity information -relevance to current tasks / team member -available task resources -priority of current task activity -operators’ availability (progress, workload, tasking availability) -operators’ interruptibility (status, cognitive availability) This information helps team members address the questions: -when / how can I help somebody else? -when / how can I be helped? -how do my actions impact others’ activities?

Relevant Activity Awareness Information is Role Dependent COM and mCDR need status and workload information of the UAV Ops to ensure individuals and team are not overwhelmed -mCDR can help UAV Ops prioritize task activities or request support from external resources -helps COM distributed TST and BDA assignments among UAV Ops UAV Ops need operator & resource availability and interruptibility information -helps minimize disruptions when someone is involved in critical tasking -helps operators know who can feasibly help them, or who might need their help

Supporting Team Supervision: Status and Workload Information on a Situation Display for Battlefield Context

Supporting Team Supervision: Status and Workload Information on a Team Status Display Z1 OK CONVOY 5% Z1 UAV OP 1 UAV OP 2 UAV OP 3 1B 1D 1C 1A 2B 2D 2C 2A 3B 3D 3C 3A Z2Z3 ISR route: total: OP 1OP 2OP 3total 10%25%10%15% 20%50%90%53% targets ISR STR 5 3 BDA:2 Comms Connectivity Status AWACSSATELLITE STRIKE TEAM UAV TEAM

Ongoing Work: Display Design & Implementation Iterating on the status and workload information displays Developing activity awareness displays to integrate into our existing UAV Op simulation environment, MAUVE, that provides operator & resource availability and interruptibility information MAUVE Tactical Display MAUVE Decision Support Display

Ongoing: Developing Team Testing Lab 3 Large-screen displays (A) 4 Reconfigurable operator stations (B) 2 Mobile operator stations (tabletPCs & handhelds) (C) Wireless headsets with microphones for ‘always on’ audio via Internet software HAL Complex Operation Center

Ongoing: Developing Team Testing Lab 3 Large-screen displays (A) 4 Reconfigurable operator stations (B) 2 Mobile operator stations (tabletPCs & handhelds) (C) Wireless headsets with microphones for ‘always on’ audio via Internet software HAL Complex Operation Center

Project Summary Enhancing activity awareness in collaborative TST environments seems to help promise for helping teams coordinate their actions and help reduce the costs of collaborating via technology Overarching goal of this project is to understand the tradeoffs between supporting individual operator performance and team performance to provide overall mission effectiveness during TST operations

Thanks! This research is funded by: Boeing Phantom Works & Air Force Research Lab For more information: or