Advanced Decision Architectures Collaborative Technology Alliance Five Lessons Learned in Human-Robot Interaction Patricia McDermott Alion Science and Technology Jennifer Riley, Ph.D. SA Technologies Douglas Gillan, Ph.D. North Carolina State University Laurel Allender, Ph.D., Army Research Laboratory

Advanced Decision Architectures Collaborative Technology Alliance RESEARCH OBJECTIVES Investigate the impact of key technological, human, and task factors on human robot interaction in a range of robotic tasks. Bandwidth reduction Span of control Unmanned vehicle characteristics Level of automation Interface designs Collaborative technologies Teamwork configurations Sense Process UnderstandPresent Communicate Helping the soldier…

Advanced Decision Architectures Collaborative Technology Alliance Research Overview Integrated HRI Research Findings Candidate Robot Interfaces Collaborative Research Experiments HRI Design Guides/ Implications Identified HRI Research Gaps Research Road Map SA Technologies Alion NMSU NCSU ARL

Advanced Decision Architectures Collaborative Technology Alliance Top 5 Signs You have Made an HRI Error 5. The operator keeps covering the same ground over and over.

Advanced Decision Architectures Collaborative Technology Alliance Perception, Navigation, and Operating Robots Field of View studies –Widening the view is most beneficial –If can’t widen view, 3 rd person view can aid navigation Screenshots of conditions in FOV/Camera Perspective study Wide FOV/3 rd Person Perspective Wide FOV/1 st Person Perspective Narrow FOV/3 rd Person Perspective Narrow FOV/1 st Person Perspective

Advanced Decision Architectures Collaborative Technology Alliance Limited Bandwidth Spatial vs. Temporal Resolution –Participants selected higher spatial resolution, were faster to identify friend/foe –If high spatial resolution not available, temporal resolution may be just as useful Perception, Navigation, and Operating Robots High Spatial Resolution High Temporal Resolution

Advanced Decision Architectures Collaborative Technology Alliance Perception, Navigation, and Operating Robots Lesson 5: Removing the human from the environment to be navigated disrupts the perception of visual information about the environment and the motion. The HRI interface can be designed to reduce this disruption and enhance the operator’s ability to navigate a robot through its distant environment and to perform the desired tasks. –Widen the field of view (FOV) if possible. Providing a third-person perspective also benefits navigation, the benefits are additive. –Temporal resolution may be sacrificed if the resolution is high and the robot is traveling at relatively low speeds. A high spatial resolution aids in target detection and identification.

Advanced Decision Architectures Collaborative Technology Alliance Top 5 Signs You have Made an HRI Error 4. The operator keeps turning the wrong direction.

Advanced Decision Architectures Collaborative Technology Alliance Team Factors and Processes Collaboration Experiments –Location of teammates –Collaborative technologies –Communication tools View from UAV View from UGV Map with player locations

Advanced Decision Architectures Collaborative Technology Alliance Team Factors and Processes Lesson 4: When robotic operators are distributed from Soldiers who need the information from robotic assets to complete a mission, the interfaces and collaborative technologies must support the communication of high quality spatial information in relation to the Soldier in the field.

Advanced Decision Architectures Collaborative Technology Alliance Team Factors and Processes The following solutions can be implemented to aid collaboration among humans with robotic tasks: –When robotics team members cannot be co-located, it is advantageous to incorporate aspects of co-location such as the ability to share imagery. –When sharing imagery, be sure to include enough reference information so that the recipient understands where the picture was taken in reference to his or her location. –Allow both team members to have full-time access to a map of the area without having to search between displays. –Allowing team members to communicate visually on a shared map can enhance the communication of highly spatial information.

Advanced Decision Architectures Collaborative Technology Alliance Top 5 Signs You have Made an HRI Error 3. The operator keeps misidentifying targets.

Advanced Decision Architectures Collaborative Technology Alliance Multitasking High workload task - integrating and interpreting visual information from the different sources –Unique shape and color cues had a huge impact on performance

Advanced Decision Architectures Collaborative Technology Alliance Multitasking and the Role of Automation Lesson 3: Operating a robot inherently involves multitasking and task switching. When designing automation to aid multitasking, designers must consider how social and human factors in HRI can interact to negatively counter the benefits of automated functions.

Advanced Decision Architectures Collaborative Technology Alliance Multitasking and the Role of Automation When implementing automation in robotic tasks, designers should keep in mind the following: –At natural break points in tasks or during down time, the system could prompt participants to switch tasks or robots. This would facilitate appropriate task switching strategies that produce better performance without sacrificing SA –Provide critical cues to help operators reconcile imagery from different perspectives such as aerial and ground or from two different ground perspectives. Shape and color were the most salient cues followed by texture, markings, and shadows. –Evaluate the coordination costs of two operators sharing one automated asset. The costs may outweigh the benefits, especially in an ad-hoc team.

Advanced Decision Architectures Collaborative Technology Alliance Top 5 Signs You have Made an HRI Error 2. The operator does not notice a decrement in the robot status

Advanced Decision Architectures Collaborative Technology Alliance SA Design Needs Lesson 2: Factors associated with human control of robotics assets can differentially impact SA. It is critical to determine the information needs of Soldiers within a context and to support the multiple facets of SA through system and interface design. In order to provide information at the right level, start with a goal-directed (cognitive) task analysis and behavioral and communications analysis.

Advanced Decision Architectures Collaborative Technology Alliance SA-Related Design Needs Support for visually demanding tasks. Examples include object recognition support, strategies or interface support for depth perception and judging size and distance, and support for parallel processing and divided attention. Support for understanding robot localization and situated-ness. Examples include improved GPS for robots, mapping tools at the interface, interface designs that present data on orientation and robot configuration, and mission recording and playback for quick SA update. Improved interface design for developing higher level SA. For example, conduct SA-oriented design of interfaces to facilitate integration of data at the interface and design for understanding of robot automation states and modes).

Advanced Decision Architectures Collaborative Technology Alliance Top 5 Signs You have Made an HRI Error 1. Your paper only includes performance data.

Advanced Decision Architectures Collaborative Technology Alliance HRI Measures Performance Data SA Data Latency experiment example –Better control performance with no latency –Decreased SA errors with latency Impacts interpretation in the context of complex tasks and goals

Advanced Decision Architectures Collaborative Technology Alliance HRI Measures Lesson 1: Performance measures and SA measures should be examined together to get the “whole” story on operational effects in HRI. Implication: There are multiple methods to assess SA: Direct assessment Subjective assessment Process assessment Result: Better understanding of interrelationships between decision making performance and SA in HRI

Advanced Decision Architectures Collaborative Technology Alliance Implications for Future Army Ways in which humans and robots can work seamlessly together in naturalistic environments Implications for display design, interaction design, team configuration, collaborative technologies, and HRI metrics Future research –Development of system design requirements –Examination of interrelationships in HRI, i.e., spatial awareness and SA, spatial awareness and mental map development –More studies should be done on the interrelationships among human performance measures such as performance time and accuracy, SA, teamwork, and collaboration

Advanced Decision Architectures Collaborative Technology Alliance SPECIAL ISSUE OF JOURNAL OF COGNITIVE ENGINEERING AND DECISION MAKING Improving Human-Robot Interaction in Complex Operational Environments: Translating Theory into Practice Topic Areas Sensor Interpretation & Integration Manipulation Navigation Planning Multiple Robot Operations Team Performance Trust and Acceptance Technological Issues Research Issues Key Dates 15 October: notification of plan to submit 15 November: paper submission Special Issue Guest Editors Jennifer M. Riley SA Technologies Patricia L. McDermott Alion Science & Technology Douglas J. Gillan North Carolina State University