1. 1 Aura Project © 2001 Carnegie Mellon University Wearable Computer Architecture and Applications Daniel P. Siewiorek Carnegie Mellon University October 30, 2001 Boeing

2. 2 Aura Project © 2001 Carnegie Mellon University

3. 3 Aura Project © 2001 Carnegie Mellon University Five Generations of Wearable Computers Navigator 2 used for aircraft maintenance VuMan 1 VuMan 2 Navigator 1 VuMan 3Navigator 2 Left, a look through the head mounted display. The user not only sees the aircraft maintenance interface, but also their work environment.

4. 4 Aura Project © 2001 Carnegie Mellon University

5. 5 Aura Project © 2001 Carnegie Mellon University Wearable Applications and Architecture l Procedures - upload at completion l Work Orders - incremental updates l Collaboration - real time interaction »Client-Server –Thin Client Legacy Systems –Interactive Electronic Technical Manuals (IETMs)

6. 6 Aura Project © 2001 Carnegie Mellon University Time Rate of Change of Data Taxonomy l Procedures. Maintenance and plant operation applications are characterized by a large volume of information that varies slowly over time. l A typical request consists of approximately ten pages of text and schematic drawings. Changes to the centralized information base can occur on a weekly basis.

7. 7 Aura Project © 2001 Carnegie Mellon University Savings Using Tactical Information Assistants in Marine Heavy Vehicle Maintenance Current Practice SAVINGS FACTOR VuMan 3 Field Trials Personnel 2:1 Inspection time 40% less SAVINGS FACTOR

8. 8 Aura Project © 2001 Carnegie Mellon University Final SystemMock-up System Prototype System Initial visit Story Boards Month 01234 Four Month Design Cycle

9. 9 Aura Project © 2001 Carnegie Mellon University Time Rate of Change of Data Taxonomy (continued) l Work Orders. The trend is towards more customization in systems. l Manufacturing or maintenance personnel receive a job list that describes the tasks and includes text and schematic documentation. This information can change on a daily or even hourly basis.

10. 10 Aura Project © 2001 Carnegie Mellon University User Interface Screen

11. 11 Aura Project © 2001 Carnegie Mellon University Time Rate of Change of Data Taxonomy (continued) l Collaboration. An individual often requires assistance. In a “Help Desk” an experienced person is contacted for audio and visual assistance. The Help Desk can service many people simultaneously. l Information can change on a minute-by- minute and sometimes even a second-by- second basis.

12. 12 Aura Project © 2001 Carnegie Mellon University Integrated Technical Information for the Air Logistics Centers (ITI-ALC) Technology Demonstration

13. 13 Aura Project © 2001 Carnegie Mellon University F-15 Depot maintenance

14. 14 Aura Project © 2001 Carnegie Mellon University Problem l Outdated, cumbersome maintenance information capability »Paper-based products »Independent, uncoordinated computer information systems

15. 15 Aura Project © 2001 Carnegie Mellon University ITI-ALC Technology Demonstration Architecture Inspector Pentium Laptop Inspection Collaboration OracleDB Windows NT Server Mechanic Wireless LAN Engineer’sWorkstation

16. 16 Aura Project © 2001 Carnegie Mellon University Evaluation & Inventory 1. Select aircraft 2. Select region 3. Pick-up 173’s 4. Check freq. defect list 5. Get tools 6. Conduct inspection 7. Check-off defect list 8. Stamp 173’s 9. Write up new defects 10. Access parts info. 11. Access TO’s 12. Access HowMal codes 13. Write new defects in U-book 14. Stamp U-book 15. Enter data into database Current ITI-ALC 1. Login 2. Hangar 3. 173 List 4. 173 Signoff 5. 173 History 4. Record Defects 5. Add New Defect 6. Submit Defects 7. Defect History

17. 17 Aura Project © 2001 Carnegie Mellon University

18. 18 Aura Project © 2001 Carnegie Mellon University

19. 19 Aura Project © 2001 Carnegie Mellon University

20. 20 Aura Project © 2001 Carnegie Mellon University Engineering Assistance 1. Mechanic finds skin defect 2. Mechanic obtains Form 202 3. Fills in fields of Form 202A 4. Views tech. data 5. Makes a rough sketch 6. Form to Scheduler 7. Form to Planner 8. Form to Engineering 9. Engineer reviews Form 202A 10. Engineer researches problem 11. Engineer goes to hangar for visual 12. Engineer fills in Form 202B 13. Reverse routing/logging above 14. Mechanic reads 202B 15. Mechanic is ready to enact repair 1. Login Screen 2. Hangar Screen 3. Form 202A 4. Confirmation Screen 3a.Parts Screen 3d. Sound Tool 3c. Sketch Tool 3b. Take Picture Current ITI-ALC 1. ELogin3. Form 202B4. Confirmation Screen 2. 202B Selection Screen

21. 21 Aura Project © 2001 Carnegie Mellon University

22. 22 Aura Project © 2001 Carnegie Mellon University Wearable PC Runs Inmedius Web-based IETM Software Server Web Server IETM Engine Client database schema Web Browser database schema IETM Authoring incremental updates Authoring Incremental Distribution Web Runtime incremental updates F/A-18 Fleet Support Organization Fleet Operating Sites IETM Authoring/Maint. Organizations

23. 23 Aura Project © 2001 Carnegie Mellon University IETM Display by Mobile Computer - IBM Concept Model is an Example ThinkPad 560X Equivalent High Spec » Full Function Portable PC in IBM High Density Package Ultimate Portability » Headphone Stereo Size System Unit, 2/3lbs (299g) IBM MicroDrive » 1" Disk, 5mm Thickness, 20g, 340 MB Capacity Transparent Head Mount Display » Invented by IBM T.J. Watson Research

24. 24 Aura Project © 2001 Carnegie Mellon University F-18 Inspection Application: Production vest fits under “Float Coat”

25. 25 Aura Project © 2001 Carnegie Mellon University 11 Mbs wireless LAN connects Wearable Computer to server

26. 26 Aura Project © 2001 Carnegie Mellon University Selection of “hot links” with CMU’s Wheel/Pointer

27. 27 Aura Project © 2001 Carnegie Mellon University Wireless and Handheld Andrew

28. 28 Aura Project © 2001 Carnegie Mellon University Academic and Administrative Buildings Residence Halls, Parking, etc Wireless Campus as of June 2000 300+ basestations

29. 29 Aura Project © 2001 Carnegie Mellon University Aura Thesis The most precious resource in computing is human attention Aura Goals reduce user distraction trade-off plentiful resources of Moore’s law for human attention achieve this scalably for mobile users in a failure-prone, variable-resource environment

30. 30 Aura Project © 2001 Carnegie Mellon University Context Aware Computing l Applications that use context to provide task- relevant information and/or services l Context is any information that can be used to characterize the situation of an entity (person, place, or physical or computational object) l Contextual sensing, adaptation, resource discovery, and augmentation l Examples of Context Aware applications »Matchmaking »Proactive assistant

31. 31 Aura Project © 2001 Carnegie Mellon University Example Agents l Notification Agent »Alert a user if they are passing within a certain distance of a task on their to do list. l Meeting Reminder Agent »Alerts a user if they are in danger of missing a meeting. l Activity Recommendation Agent »Recommends possible activities/meetings that a user might like to attend based on their interests.

32. 32 Aura Project © 2001 Carnegie Mellon University Context Aware Computing Platform: The Spot Architecture

33. 33 Aura Project © 2001 Carnegie Mellon University Spot Wearable Computer

34. 34 Aura Project © 2001 Carnegie Mellon University Belt Worn Spot and Head Mounted Display

35. 35 Aura Project © 2001 Carnegie Mellon University Research Challenges User interface models—new application metaphors require experimentation Input/output modalities—accuracy and ease of use Quick Interface Evaluation Methodology—to use during design Match capability with application—resist “highest performance” temptation