1. April 2003 Atacama Expedition HCI Field Research Report Angela Wagner Kristina McBlain

2. HCI Field Report: April 2003 Atacama Expedition Slide: 2 of 30 Project Description Introduction  Human-Computer Interaction (HCI) Team  Kristina McBlain MHCI (2003) Human Computer Interaction Institute School of Computer Science Carnegie Mellon University  Angela Wagner MHCI Student Human Computer Interaction Institute School of Computer Science Carnegie Mellon University

3. HCI Field Report: April 2003 Atacama Expedition Slide: 3 of 30 Project Description Introduction  Research Goal  Understand the complete system for human monitoring of the robot during autonomous navigation and make recommendations for system improvements.  Research Foci  Operator Interface (OI) Graphical User Interface (GUI)  OI GUI usage patterns  Robot monitoring system including electronic and “Human to Human” communication  Field process usability  Research Sites  Pittsburgh, PA, USA  Atacama Desert, Chile (April 13 - April 17)

4. HCI Field Report: April 2003 Atacama Expedition Slide: 4 of 30 Data Collection Pittsburgh  Use of Existing Operator Interface  Interviewed Operator Interface GUI developer  Interviewed OI GUI users from Arctic expedition (2001)  Observed use of OI GUI  Documented current OI GUI  Complete System Structure  Studied interaction of system processes

5. HCI Field Report: April 2003 Atacama Expedition Slide: 5 of 30 Data Collection: Operator Interface Pittsburgh

6. HCI Field Report: April 2003 Atacama Expedition Slide: 6 of 30 Data Collection Atacama  Autonomous Navigation Observation  Observe experienced team members monitoring operator interface during autonomous navigation  Observe team members observing traversing robot  Observe code integration process  Video tape interface usage during autonomous navigation as available (8 hours)  Use “think aloud” technique during observations  Identify communication pathways  Identify additional tasks occurring in parallel

7. HCI Field Report: April 2003 Atacama Expedition Slide: 7 of 30 Environment During Observation Atacama  Hyperion’s Team’s Navigation Goals  Code integration and debugging  Autonomous navigation  Limit contact between OPS tent and robot observers  Field Team Deployment Pattern  OPS tent: 1 operator interface monitor and field communication contact, 1 mission planner developer, 1 health monitor developer  Robot: 2-3 robot observers, 1 field communication contact

8. HCI Field Report: April 2003 Atacama Expedition Slide: 8 of 30 Environment During Observation Atacama Operations TentIn Field with Robot

9. HCI Field Report: April 2003 Atacama Expedition Slide: 9 of 30 Environment During Observation Atacama  Portable HCI Lab  Video camera recording ops tent communication and operator interface GUI  Note taking: laptop and notebooks  Food tent conference center  4-Runner workstation  Digital camera

10. HCI Field Report: April 2003 Atacama Expedition Slide: 10 of 30 Data Analysis Post Data Collection  Video Coding  View tapes and record significant events, providing traceable support for recommendations  Event Types Coded  Communication: electronic and verbal  Tool use: operator interface and others  Operator Interface: breakdowns and redundancies  Communication Modeling  Tracked electronic communication between system processes and operator interface during autonomous navigation  Tracked verbal communication within ops tent and between ops tent and field reps during autonomous navigation

11. HCI Field Report: April 2003 Atacama Expedition Slide: 11 of 30 Autonomous Navigation: Electronic Communication Data Analysis Robot OI GUI Health Monitor Mission Planner Navigator Vehicle Controller Mission Executive State Estimator Stereo Mapper Micro-raptor CVS

12. HCI Field Report: April 2003 Atacama Expedition Slide: 12 of 30 Autonomous Navigation: Electronic / Human Comm. Data Analysis Robot OI GUI Health Monitor Mission Planner Navigator Vehicle Controller Mission Executive State Estimator Stereo Mapper Micro-raptor CVS OI Operator (Engineering) Mission Planner (Science)

13. Legend HCI Field Report: April 2003 Atacama Expedition Slide: 13 of 30 Autonomous Navigation: All Communication Data Analysis Robot OI GUI Health Monitor Mission Planner Navigator Vehicle Controller Mission Executive State Estimator Stereo Mapper Micro-raptor CVS Mission Planner Debug & Develop MP Code Monitor Ops Tent Comm. Lead Code Integration Coordinator Hardware Troubleshooting Assistant Data Collection Configuration Lead Health Monitor Debug & Develop HM Code Monitor Robot Situation Observers (1-3) Robot Observer Communication Lead Robot Hardware Troubleshooting Lead Camera Data Collection Lead Micro-raptor Expert 2-way Radio Dennis Cam Experiment Set up and Data Collection OI Operator (Engineering) Mission Planner (Science) Process arbiter = expected human role = not expected human role = not expected device role = human-human interaction = human-software interaction = human-hardware interaction

14. HCI Field Report: April 2003 Atacama Expedition Slide: 14 of 30 Assumptions: Long Term Goals Motivation  Transparent Robot  Robot operations become transparent and science data collection becomes focus  Robot system developers are not routinely involved in experiments  Non-Specialist Operator Interface Monitor  Operator Interface monitors are not developers of the system or specialists in robot’s system  Operator Interface monitors have technical experience and have received some training in robot’s system

15. HCI Field Report: April 2003 Atacama Expedition Slide: 15 of 30 Assumptions: Near Term Goals Motivation  Standardized Testing Procedures  Rely on electronic communication (primarily OI) to monitor robot and troubleshoot  Create redundancies in specialist knowledge of software and hardware  Code Maintenance During Field Trials  Create shared knowledge base of code structure and function  Customize information tracking tools to facilitate troubleshooting and development

16. HCI Field Report: April 2003 Atacama Expedition Slide: 16 of 30 Foci Recommendations  OI GUI  Organization  Consolidate tools  Update data displayed  Functional breakdowns  Field Process Usability  Verify field process accommodates anticipated usage

17. HCI Field Report: April 2003 Atacama Expedition Slide: 17 of 30 OI GUI: Organization Recommendations  Expected  Observed Second GUI with camera data (stereo)

18. HCI Field Report: April 2003 Atacama Expedition Slide: 18 of 30 OI GUI: Organization: Decision Points Recommendations  Decision Point 1: User Profile/s  High level  More granular  Do these profiles / roles change full project timeframe?  Decision Point 3: User Location (Remote or co-located (with other users and with robot))  Decision Point 2: Information needed by each User Profile at each timeframe in project  Decision Point 4: GUI environment configuration (1 monitor or multiple, OS, GUI dev decision/resulting advantages and constraints)

19. HCI Field Report: April 2003 Atacama Expedition Slide: 19 of 30 OI GUI: Organization: Near Term Recommendations  Robot Monitoring with Concurrent Development  Plan for development needs  Define roles  Define “expected” tasks  Plan to rely on electronic communication and orient interface to autonomous navigation  Improve existing tool usability and function  Consolidate tools  Allocate screen space for non- OI tools  Update data displayed  Address identified functional breakdowns  Deprecate use of interface components that may not be feasible with longer-term remote bandwidth limitations  Second computer / monitor

20. HCI Field Report: April 2003 Atacama Expedition Slide: 20 of 30 OI GUI: Near Term: Consolidate Tools Recommendations  Integrate Driving Tool and Kinematics Tool  Combine all visual representations of rover state  Transform driving commands into collapsible component of new integrated tool  Display current roll and pitch limits in addition to current roll and pitch  Consider creating easy way to access average rover speed in addition to commanded (target) rover speed for use in mission planning.  Consider eliminating or hiding infrequently used data.

21. HCI Field Report: April 2003 Atacama Expedition Slide: 21 of 30 OI GUI: Near Term: Consolidate Tools Recommendations  Allocate Screen Space for Non-Operator Interface Tools  Other tools (Micro-Raptor, telemetry manager log) are being used on a different desktop display on the same screen that the operator interface is being displayed, causing the operator interface monitor to miss key troubleshooting information  Include another physical screen for viewing these tools  Prioritize GUI windows to allow these tools coexist on one screen with the operator interface

22. HCI Field Report: April 2003 Atacama Expedition Slide: 22 of 30 OI GUI: Near Term: Consolidate Tools Recommendations  Create Stereo Camera Image Tool  Users monitoring the operator interface rely heavily on the stereo camera image in “Chris’ Tool”  Under tighter communication constraints, stereo camera image displays may not be possible  Terrain Sensor already displays stereo map and stereo disparity, remove redundancy with “Chris’ Tool”  Include a panel to only display stereo camera image to allow the removal of stereo camera image data without loosing stereo map and stereo disparity  Goal of one OI GUI for each expedition.

23. HCI Field Report: April 2003 Atacama Expedition Slide: 23 of 30 OI GUI: Near Term: Update Data Displayed Recommendations  Incorporate Mission Planner Data  Include Mission Planner state in Operator Interface including Planning, Executing Plan, and Out of Contact  Include Mission Planner data in Local Map Tool  Display time last plan was uploaded  Display current waypoint and total number of waypoints  Display time last waypoint was reached  Remove Unused Information  Remove Sun Yaw in Local Map Tool Heading Indicator  Remove “Laser Goodness Map” in Terrain Sensor Tool  Remove unnecessary Health Monitor faults

24. HCI Field Report: April 2003 Atacama Expedition Slide: 24 of 30 OI GUI: Near Term: Update Data Displayed Recommendations  Redesign Health Monitor Interface  Logically organize the faults to facilitate error handling  Adapt the look and feel of the faults to facilitate trigger detection and clarify enabled vs. disabled faults  Allow customizable view to decrease screen clutter and data redundancy and minimize cognitive load of operator interface monitor  Change placement of and access to “Change Fault Constraint”  Remove faults connected to processes which are no longer used in the system

25. HCI Field Report: April 2003 Atacama Expedition Slide: 25 of 30 OI GUI: Near Term: Update Data Displayed Recommendations  Redesign Health Monitor Interface  Provide time stamped message indicating the most recent data refresh and current status (Is the Health Monitor “alive” and in sync with OI)  Create log of triggered and un-triggered faults with time stamp to enable review of missed triggers  Provide access to telemetry log data relating to triggered fault condition  Consider audio alert for e-stop if usage pattern will involve moving between screens in a manner that may hide the visual cue.

26. HCI Field Report: April 2003 Atacama Expedition Slide: 26 of 30 OI GUI: Near Term: Functional Breakdowns Recommendations  Improve E-stop Button Function  E-stop release on single click  Provide Operating Mode Status  Include error message when Operating Mode change is not possible  Include status message when Operating Mode is changed by a process  Enhance E-stop Information  Display e-stop sender and time stamp in main OI tool

27. Window 1: displayed information HCI Field Report: April 2003 Atacama Expedition Slide: 27 of 30 OI GUI: Near Term: Organization Recommendations Main robot monitoring section: kinematics, stereo cam, local map E-Stop: source Health Monitor Access log for e-stop Mission Planner Window 2: displayed information Left Stereo ImageComment Log Select item drop-down E-stop history Drive Tool modes

28. HCI Field Report: April 2003 Atacama Expedition Slide: 28 of 30 OI GUI: Organization: Longer Term Recommendations  Science and OI interface integration  Non-developer OI use  Increase sophistication of data visualization  Make “the windshield” a reality

29. HCI Field Report: April 2003 Atacama Expedition Slide: 29 of 30 Field Process Usability: Verification Recommendations  Manage Software Interdependencies  Document interaction between processes  Implement process for reverting to most recent, stable version of code quickly  Manage Inter-Process Communication  Create an IPC message tracking system to reduce debugging time and facilitate code management  Create process to verify needed research data is being captured  Manage Process Starting and Stopping  Provide system checks for process boot sequence and enhance efficiency of re-boot

30. HCI Field Report: April 2003 Atacama Expedition Slide: 30 of 30 Questions ? ? ?