Collaborative Work Systems, Inc CWS Collaborative Work Systems, Inc Geo-Docent: Improving Human, Team, and Organizational Performance with Geographically Triggered Multimodal Information Roger J. Chapman, PhD President Collaborative Work Systems (CWS), Inc LTC James Merlo, PhD Academy Professor US Military Academy

Collaborative Work Systems, Inc © Copyright CWS, Inc 2007 CWS Proprietary Information – Do not distribute without permission of the author 2

Collaborative Work Systems, Inc © Copyright CWS, Inc 2007 CWS Proprietary Information – Do not distribute without permission of the author Geo-Docent: A system for authoring & utilizing geographically triggered multimodal information 1. Author geotriggered presentation of multimodal information 2. Distribute to users 3. Run in real-world or virtual mode 4. Log GPS track & events enroute 5. Review The map can be directly annotated to create “trigger spots/regions” With a GPS device hotspots can automatically be created for the author’s current position. The set of geocondition -action rules Note: The mobile computing device used during run mode need not necessary be the same as used for authoring. When the user’s actual or simulated geoposition triggers a rule the associated multimodal content is executed Logged track & events facilitate an After Action Review 3 Potential applications include mixed reality training, controlled field experiments, patrol data collection, handoffs in context, accident investigation, and guided navigation through complex areas of operation.

Collaborative Work Systems, Inc © Copyright CWS, Inc 2007 CWS Proprietary Information – Do not distribute without permission of the author 4 Author geotriggered multimodal information presentation The map can be directly annotated to create “trigger spots/regions” With a GPS device hotspots can automatically be created for the author’s current position to support “in the field” authoring. If the author is using a mobile device with a built-in camera or a separate camera that time-stamps photographs these can be automatically linked to the hotspot. The set of geocondition-action rules have the pattern: (1) geocondition (e.g. on entry, on exit, on crossing, within range x), (2) action (e.g. play audio, play video, popup message, activate new geotriggered map), and (3) rule specific parameters (e.g. audio filename) The set of geotriggered mapsMap annotation options

Collaborative Work Systems, Inc © Copyright CWS, Inc 2007 CWS Proprietary Information – Do not distribute without permission of the author5 Photographs may be annotated in Geo-Docent Options for synchronized voice recording and drawing on each photograph

Collaborative Work Systems, Inc © Copyright CWS, Inc 2007 CWS Proprietary Information – Do not distribute without permission of the author6 Running in real-world or virtual mode In real-world mode when the user’s actual geoposition triggers one or more rules the associated multimodal content is executed In test mode a virtual GPS location can be placed anywhere on the map to trigger the associated content The Geo-Docent content is shared as single file that can be delivered via the Web or etc., and then loaded into Geo- Docent

Collaborative Work Systems, Inc © Copyright CWS, Inc 2007 CWS Proprietary Information – Do not distribute without permission of the author7 Logging GPS tracks & events to support AAR The built-in GPS Tracker module supports logging the users current location over time to help facilitate an AAR. Note: Geo-Docent data can be exported to Google Earth to provide a context-rich environment for this AAR and for sharing the results with non-Geo- Docent users.

Collaborative Work Systems, Inc © Copyright CWS, Inc 2007 CWS Proprietary Information – Do not distribute without permission of the author8 Quick Demo

Collaborative Work Systems, Inc © Copyright CWS, Inc 2007 CWS Proprietary Information – Do not distribute without permission of the author Wearing communications technology output displays can improve user efficiency

Collaborative Work Systems, Inc © Copyright CWS, Inc 2007 CWS Proprietary Information – Do not distribute without permission of the author With multiple tactors it is relatively simple to interpret tactile displays containing geospatial information 10 A direction to move in Directional information is represented relative to the body. Other information must be encoded as the type of pulse or pattern of pulses.

Collaborative Work Systems, Inc © Copyright CWS, Inc 2007 CWS Proprietary Information – Do not distribute without permission of the author Databases with geotagged information can be utilized in the field with GPS enabled location “triggers”  DARPA’s TiGR system has geotagged patrol collected data that could be used to create dynamic walkthroughs or spontaneous feedback as the Geo-Docent user explores areas of operation in simulation or real “in the field” mode - Raw data (could be presented by text-to-speech and/or multimedia)  Geotagged event summaries, often support by photos  Patrol routes - Aggregated data  Warning of entry to categorized regions 11

Collaborative Work Systems, Inc © Copyright CWS, Inc 2007 CWS Proprietary Information – Do not distribute without permission of the author Conclusions  The Geo-Docent prototype demonstrates the basic feasibility of integrating GPS and Tactile technologies to support the creation and execution of geotriggered Tactile messages  Further development is needed to create a robust, complete, and field testable system - Cadets at USMA conducting human factors systems analysis  Participants authoring geotriggered content for USMA  Participants using geotriggered content to receive a tour of USMA - Can use cases be developed that suggest it would support any of the following applications:  mixed reality training, controlled field experiments, patrol preparation, patrol data collection, handoffs in context, accident investigation 12