1. Thrust III HRI Studies for Human Factors and Performance Evaluation J. A. Adams (Lead – Task A) P. Hinds (Lead – Task B) C. Breazeal J. How N. Roy MURI 8 Kickoff Meeting 2007

2. MURI 8 Kickoff Meeting 2007 Thrust III: Objective  Empirical evaluation of remote human operator and human teammate levels of cognitive-load, situation awareness, trust, and performance related to efficient, flexible, and effective HRI, human-robot team structures, team mental models, and system scalability. MIT-Vanderbilt-Stanford UW-UMASS Amherst

3. MURI 8 Kickoff Meeting 2007 III.A: Task Objectives  Provide a means of empirically evaluating potential human-robot interaction and visualization options prior to full system and hardware integration. MIT-Vanderbilt-Stanford UW-UMASS Amherst

4. MURI 8 Kickoff Meeting 2007 III.A Existing Technology  Microsoft Robotics Studio simulation of the MDS robots  Quadrotor simulator MIT-Vanderbilt-Stanford UW-UMASS Amherst

5. MURI 8 Kickoff Meeting 2007 III.A Existing Evaluation Methods  A number of evaluation methods exist that will be incorporated, as appropriate, for assessing  Usability (e.g. Errors)  Workload (e.g. NASA TLX, Multiple Resources Questionnaire)  Situation Awareness (e.g. SAGAT, SART)  Team coordination (e.g. Observation, Efficiency)  HRI Metrics (e.g. Effectiveness, Fan Out) MIT-Vanderbilt-Stanford UW-UMASS Amherst

6. MURI 8 Kickoff Meeting 2007 III.A Technical Advances  Quantitative evaluation of visualization interaction alternatives with simulated vehicles.  Permits preliminary evaluation for cognitive limitations, situation awareness, and team performance.  Permits the narrowing of alternatives for future development. MIT-Vanderbilt-Stanford UW-UMASS Amherst

7. MURI 8 Kickoff Meeting 2007 III.A: Year 1 Milestones/Deliverables  Simulation based evaluations of potential UV specialist, triage officer (division chief), and incident commander.  Visualizations and interactions  Transitions with historical context MIT-Vanderbilt-Stanford UW-UMASS Amherst

8. MURI 8 Kickoff Meeting 2007 III.A: Year 2+ Milestones/Deliverables  Simulation based evaluations of  triage specific HRI for the UV specialist, triage officer (division chief), and incident commander.  Situation awareness drill down  HAZMAT HRI capabilities.  the integrated HRI visualizations, interactions, drill down, and transition capabilities. MIT-Vanderbilt-Stanford UW-UMASS Amherst

9. MURI 8 Kickoff Meeting 2007 III.B: Task Objectives  Empirical evaluation of human operator and human team member levels with real hardware.  Assessing cognitive-load, situation awareness, trust, and performance related to efficient, flexible, and effective HRI, human-robot team structures, team mental models, and system scalability. MIT-Vanderbilt-Stanford UW-UMASS Amherst

10. MURI 8 Kickoff Meeting 2007 III.B Existing Evaluation Methods  Trust  An elusive characteristic to measure in human-robot interaction.  Neither human-human nor trust in automation scales are effective indicators of trust in embodied and mobile robots. MIT-Vanderbilt-Stanford UW-UMASS Amherst

11. MURI 8 Kickoff Meeting 2007 III.B Technical Advances  Empirical results that advance the state of the art understanding of  trust by humans in robotic systems  teaming between humans and robotic systems  situation awareness and cognitive demands placed on humans  functioning as remote supervisors of robotic and human teams  transitioning between remote operator roles  maintaining common ground among the multiple user levels/roles.  when acting as a peer/teammate with robotic systems MIT-Vanderbilt-Stanford UW-UMASS Amherst

12. MURI 8 Kickoff Meeting 2007 III.B: Year 1 Milestones/Deliverables  Task specific evaluations of UV specialist, triage officer (division chief), and incident commander.  Visualizations and interactions  Transitions with historical context MIT-Vanderbilt-Stanford UW-UMASS Amherst

13. MURI 8 Kickoff Meeting 2007 III.B: Year 2+ Milestones/Deliverables  Year 2: Continued evaluations  triage specific HRI for the UV specialist, triage officer (division chief), and incident commander.  Situation awareness drill down  HAZMAT HRI capabilities.  Peer-to-peer HRI capabilities  Year 2+:  Evaluation of integrated remote HRI visualizations, interactions, drill down, and transition capabilities.  System evaluations of overall and human-robot team performance, trust, and system capabilities.  Evaluations of distributed HRI. MIT-Vanderbilt-Stanford UW-UMASS Amherst