1. Intelligent Systems Division (IC  TC  TI) Collaborative and Assistant Systems (CAS) Research ROSES Partnership Opportunities Rich Keller rkeller@mail.arc.nasa.gov 2/17/05

2. Collaborative and Assistant Systems: Major Themes Human-centered systems –Work-systems design, modeling, simulation, and evaluation –Procedure design, modeling, validation, training, and execution support –Integrated field tests –Multi-modal interfaces and human / automation / robot interaction Situation awareness and decision support systems –Integrated simulations supporting what-if analysis –Contingency planning and assistance Complex information and knowledge management –Heterogeneous data storage, search, integration, and analysis –Semantic data organization, metadata management, and ontologies –Lessons learned and expertise capture Software Composition and Integration –Middleware, components, services, frameworks and portals

3. Major CAS Projects Human-centered systems –Clarissa (ISS DTO) and Advanced Astronaut Assistants –Regulus Open-source spoken dialog toolkit –Mobile Agents and Advanced EVA technologies –Optimal human interface synthesis Situation awareness and decision support systems –Simulation-based Acquisition –Intelligent Launch and Range –UAV Collaboration and Coordinated Mission Mgmt Complex information and knowledge management –Simstation and Digital Shuttle –Netmark and PMT (major rollout and EVM integration) –SemanticOrganizer for Science and Accident Investigation –Aviation data integration, mining, and trending Software Composition and Integration –CIP for MER Extended Mission –MCT Fabric demonstration for MSL

4. Human-Centered Systems Some technology pitfalls: –“Build it and they will come.” –“… but it’s a great technology.” –“Lets just ask them what they want.” Technology People & Culture Physical Environment Human-Centered Systems: A systematic methodology for designing systems that optimize the teaming of humans and machines. Methodology : Work-systems design, modeling, simulation, and evaluation Procedure design, modeling, validation, training, and execution support User-centered design Integrated field tests Crew and team organizational design and modeling Flight deck and mission operations facilities design and evaluation

5. HCC & Fatigue Countermeasures: Improved data understanding and Enhanced situational awareness CIP: Customizable data navigation, search, and information management Viz: High fidelity terrain modeling and analysis MAPGEN: Activity plan development and analysis MERBoard: Collaborative information analysis and sharing MER science and uplink team members have estimated that overall science return increased by more than 20%. Ames Capabilities for Mars Exploration Rover (MER) Mission Operations

6. Collaborative Information Portal The technology developed for the CIP project has application to the broader Information Infrastructures technology (InfoCore) research goals. The long term objective of this research is to provide a seamless information infrastructure where data from large numbers of heterogeneous information systems are easily accessible and presented in a “smart” human-centered user interface. This will enable improved decision making, reduce risk, and maximize leverage of the vast information resource and corporate knowledge developed by NASA and industry. A Web-based information management system being developed to support the surface operations phase of the Mars Exploration Rover (MER) Mission in 2004. This system will provide Mission Operations and Science Teams with rapid and intuitive access to a broad range of Mission status and planning information to assist in the daily decision making process. Integrated MER Mission Engineering & Science Data Management

7. MERBoard/Xboard Collaborative Workspace Collaborative Content Creation for Mission Planning

8. Robot Mule tracks Astronauts & takes photos when commanded Robot in “follow me” mode Voice annotation is recorded and transmitted to database in habitat & to RST on earth Astros can work fully in parallel, talking to personal agents Mobile Agents: Coordinating Human-Robot Interactions Utah Field Tests 2003 and 2004 50 Participants over 17 days 3 NASA centers & 2 universities Diverse scenarios, rough terrain 2 geologists; authentic science Analog Simulation of Mars Surface Exploration

9. Cart and instrument package Cameras: environmental pan/zoom Webcam zooming closeup sensor videocam wide-field mat still camera ScienceOrganizer Information Repository Experiment management tools Microbial mats Greenhouse server w/ intelligent agent Science team experiment design experiment scheduling automated instrument control automated monitoring automated data acquisition Remote control XYZ positioning table Greenhouse Collaboratory: A remote control scientific experimentation facility Greenhouse

10. Semantic Network overlays the SemanticOrganizer Information Repository SemanticOrganizer Interface  A customizable knowledge management system supporting distributed teams of science and engineering investigators  Digital information repository stores heterogeneous investigation documents, information, and metadata; Repository features: semantic cross-linkage for rapid access to interrelated info (e.g., evidence & hypotheses) intuitive, domain-specific terminology automated knowledge acquisition via inference  Serves as an archival record of investigation Major SemanticOrganizer Applications Early Microbial Ecosystems/ NASA Astrobiology Institute Astrobiology Center for the Study of Biosignatures Tulane/NIH Malaria Study Mars Analog Drilling Mission Simulated Human-Robot Mars Surface Exploration Mars Exploration Rovers Columbia Accident Investigation CONTOUR Spacecraft Mishap Investigation Helios UAV Crash Moffett Airshow Investigation Canard Rotor Wing Investigation Science Organizer Investigation Organizer Right side panel displays metadata for the repository node being inspected Left side panel uses semantic links to display all information related to the repository node shown on the right Scientific Knowledge Management

11. Decision Support: Intelligent Launch & Range Operations

12. Spoken Dialog Assistant Systems Problem:Problem: –Astronauts perform numerous tasks where they need to both obtain information about activities and use their hands. –Use of a keyboard in zero-gravity is cumbersome. Solution:Solution: –Ames has been developing spoken dialogue systems since 1999 to assist astronauts and allow them to interact with the computer using natural language. –Systems are able to track the dialogue to maintain context CLARISSA Flight Experiment: Intelligent Procedures and ChecklistsCLARISSA Flight Experiment: Intelligent Procedures and Checklists –Allows astronaut to interact using natural language with an intelligent procedure/checklist system. –Eliminates the need for an astronaut to read the procedure while one does the procedure. –Strong support from the astronaut office. –Path-finding flight experiment opening the door to a wide range of intelligent astronaut assistant technology.