1. The Mobile Agents 2005 Field Test at MDRS: Planning for Exploration William J. Clancey NASA Ames Research Center Chief Scientist, Human-Centered Computing Intelligent Systems Division

2. Brahms: Bill Clancey, Maarten Sierhuis, Ron van Hoof, Mike Scott, Serge Yentus MEX: Rick Alena, John Ossenfort, Charles Lee, Ed Walker, Hugo Notario RIALIST: John Dowding, Jim Hieronymus Geology: Abby Semple & Brent Garry ERA: Jeff Graham, Rob Hirsh, Eduardo Herrera, Bill Vreugde RST:Shannon Rupert + 12 participating scientists NREN:Ray Gilstrap, Kevin Bass SciOrg:Dan Berrios, Rich Keller MA 2005 Participants

3. Origins of MA Design: Historical & Analog Missions Apollo’s CapCom, a virtual assistant on the moon Biologists navigating in Haughton Crater (1999) Crew struggles with GPS during MDRS5 simulation

4. WILLIAM J. CLANCEY, PI & MAARTEN SIERHUIS, PROJECT MANAGER Human-Systems Integration Device Control: sensors/ instruments, tools, robots, systems (life, power, computing) Data Flow: Queue, sample, store, associate, share, display Work Management: Monitor, advise (loc, time, activity), interpret, alert Voice-Commanded Toolkit

5. Technical Features Multi-Agent System –Functional & personal –Distributed control & data- processing Wireless Computing –Platform independent –Tested over 5 km Modular Configurations –Plug & play (Java API) –E.g., 5 robots, 3 cameras, 5 biosensors, 2 computer sensors, dGPS, headsets Astronaut Backpack System

6. Mobile Agents Utah Field Test 2005 50 Participants; 17 days; 2 NASA centers & several universities Multi-robot, auto-recon, & freeform exploration scenarios More proactive ERA design More proactive RST More robust, faster, reusable agent system (> 60) Improved alerts, queries (> 90), & auto-associations EVA crew less reliant on HabCom crew member Geologists Exploring with Mobile Agents

7. New MA’05 Operations Interacting with the robots to determine their status: Whom are you following? Do you still have comms? Establish dependence on robot relay: Boudreaux, join me. Contextual robot move commands: Come here. Go there. Directed photos: Take a picture of work site 5. Astronaut naming of “work stations" (default 1 m areas) within work sites: Call this location 9 with a radius of 5 m. Astronauts can refer to names teammates create. Requesting locations and navigation information: Where is work site 5? Where am I? List all the locations near work site 6. Where is the next activity?

8. New ERA Hardware Instrument Arm Robotically Deployed Relay (RDR) RDR Pin Assembly Thibodeaux (2nd ERA)

9. ERA Spiral Recon Plan MDRS

10. Mobile Agents Pooh’s Corner Comms Network (2005) Rick Alena, NASA Ames,March 2005

11. Plan B: ATV with ERA Agent & Operator Display

12. Mobile Agents MDRS38 Configuration Astro Agent HabCom Agent ERA Agent ERA Hab Crew Astro RS T

13. Collaborative Interplanetary Science EVA astronaut 2 ERA Brahms VM ERA agent Comm. agent ERA MDRS Crew SpaceSuit_2 Brahms VM SpaceSuit_1 agent Astro_1 agent EVA astronaut 1 SpaceSuit_1 Brahms VM SpaceSuit_1 agent Astro_1 agent SUNY Buffalo RST Mars Society RST AZ CA RST Facilitator UK/NY Compendium MeetingReplay Hab Brahms VM CapCom agent Science Organizer agent Compendium Aagent ScienceOrganizer Aagent ScienceOrganizer Compendium Maarten Sierhuis, RIACS/NASA Ames

14. EVA Data in Science Organizer (WkStn9)

15. From EVA to Desktop From: capcom@agentisolutions.com Subject: MDRS New ImageCollection: image_collection6 Date: April 15, 2005 5:36:55 PM PDT To: mdrs_rst@agentisolutions.com Cc: mdrs_crew@agentisolutions.com, mdrs_ops@agentisolutions.com New ImageCollection: image_collection6 EVA Plan: Day_5_Astro_EVA_2_To_Beyond_ Poohs_Corner_Plan Activity: WorkInRavine Creator: AstroOneObject ID: AstroOneModel_IMAGE_COLLECTION_3 TimeStamp: 04/16/2005 01:08:07 Latitude: 3824.201417 NORTH Longitude: 11046.8688492 WEST Link for RST: https://marst.arc.nasa.gov/mdrs2005/183971... https://marst.arc.nasa.gov/mdrs2005/183971

16. Science Organizer Data Map Generated During EVA Dan Berrios, NASA Ames

17. MDRS IT Network OpsCon Win2003 Server HabObs LCD Display Monitor Display WLAN AP900+ DirectWay 56 Kbps Habcom Laptop MEX-NET WLAN Root Printer ERA Workstations MDRS Top Floor Network Shoreline Phone Processor MEX WorkStations LCD Display LCD Display Hub TP 390 MDRS Lower Network Hub MDRS WLAN DHCP SW NREN Satellite Link Network Phone Lines Hub MA Workstations Hub Science Organizer ERA WorkStations MEX-Hab(g) WLAN Root Ext Hab Wired Network Rick Alena, NASA Ames,March 2005

18. ISP/IT NETWORK (TAZ to MDRS internal hubs) MEX Net (hubs to Tropos & EVA Computers) KaOS Registry (agent locator) Mobile Agents Computers (AstroPacks, ERAs, HabCom) Agents (Personal, Loc, Nav, Plan, Comm) Peripheral Devices (dGPS, biosens, cameras, robots, db, email) Mobile Agents Systems Architecture

19. Experiment Observations (Day 5 EVA 2 Beyond Pooh’s Corner) Enables independent, simultaneous work, which geologists prefer But agents talk over astro conversations –Astro cannot hear that partner is listening to agent –Astro interleaves response with conversation –Results in “What did you say?” Ready for next level: Voice loop management, alternative feedback, protocol personalization Overall: System Responsive and processed data appropriately

20. Example Good Results Where is Boudreaux? Boudreaux, stop. How much time is left? What is the current activity? (Astro2) Where is Astronaut 1? Where am I? Warning: Time exceeded. Sorry: Can’t do that. Warning: Battery is low.

21. Some Not So Good Results Misunderstood “okay” as volume change command. Boudreaux is 1.1794857 E 7 meters North-East-East from you. Thibodeaux, join me (OK) Astro2 remarks to partner that join command worked; Astro 1 is listening to his agent and didn’t understand; Astro2 repeats what she said, but agent hears this as the command itself!

22. DesertRATS (Sept 05) Ames, JSC, Glenn @ Meteor Crater area Scout Rover = ERA 2 Pressurized Suits & Helmet-mounted display Power, pressure, coolant systems data Standard commands: Move, Go To, Photo, Print label, Where is…? EVA plan monitored Scout Rover: Mobile Agents allows voice commanding

23. For more information… http://www.marssociety.org/ MDRS/fs04 –“Mission Info” –Daily reports –Photographs http://bill.clancey.name –1st Space Exploration Conf. –Simulating Activities –Field Science Ethnography –Automating CapCom –Roles for Agent Assistants in Field Science Brahms: www.agentisolutions.com Special thanks to Abby Semple and Brent Garry

24. Basic Configuration 2004-2005

25. Types of Agents

26. Mobile Agents Extended Distance EVA Network L2 L3 L4 L5 Link L1 mex-net B269 IMT Lab Or Habitat Primary Repeater Astro 1 AstroPack1 and iPAQ ERA1 Mobile Rptr L1 - Cisco 802.11b mex-net bridge L2 - Cisco 802.11b mex-net bridge L3 - Cisco 802.11b(g) atv2-net bridge L4 - 802.11b(g) atv2-net infra L5 - 802.11b(g) atv2-net infra L6 - 802.11b(g) atv2-net infra HH ATV2 backbone Cluster root Astro 2 Full-duplex L3 alt ERA2 L5 RDR1 RDR2 L4 alt Mesh paths Rick Alena, NASA Ames,March 2005

27. MEX AstroPack Diagram iPAQ Headsets Audio WLAN link to ATV or ERA Laptop 2+ GHz NONIN Heart rate + SaO 2 WLAN 802.11g Client MEX Pack Functions Realist Voice Recognition Host Wireless LAN Client dGPS/GPS coordinate distribution Digital camera downloads Biosensors - Nonin Optional video camera Support multiple iPAQs or penpads Brahms host Video Camera (option) Explorer-worn Devices Digital Camera MEX Pack Devices dGPS WLAN link to ATV or ERA 900 MHZ Radio Modem dGPS Corrections GMR Radio Push to Talk Audio Rick Alena, NASA Ames,March 2005

29. Tracking Names 4/16/2005 1:02:49 GMT mpak-b Call my location work site ten. 4/16/2005 1:02:49 GMT Do you want to declare the area within a radius of 10 meters Work Site 10? 4/16/2005 1:42:32 GMT mpak-a Call my location work site ten. 4/16/2005 1:43:03 GMT A location with the name Work Site 10 already exists please choose a different name.

30. Crew Activity Analyzer (Foster Miller SBIR)

31. Crew Activity Analyzer Tool

32. Hanksville Schools Visit