1. Hear&There An Augmented Reality System of Linked Audio Joey Rozier Karrie Karahalios and Judith Donath Sociable Media Group MIT Media Lab ICAD 2000 Conference - April 3, 2000

2. Overview Project Purpose and Goals Brief Scenario Basic Augmentation Details Hardware Authoring and Playback Security and Privacy

3. Purpose and Goals Augment Environment with Audio –Augmented Reality (or Mixed Reality) Develop Social Augmentations –Oral Histories –Focus on Authoring Activation Networks - Linking Self-Contained and Portable Limit Use of Visual Cues

4. Example Scenario Augmented Boston Common Can hear audio “left behind” (maybe history) Guided towards related audio (moo...) Author can add his/her own sound Others are unaware

5. Augmentation Details The SoundSpot –Basic unit of augmentation –Any type of audio (voice, music, whatever) –“Placed” at a specific location (longitude/latitude) –Has a specified size –Other audio properties (braids, etc.) –User walking “into” SoundSpot hears it

6. Augmentation Details Example

7. Augmentation Details Must Determine User’s Location GPS –Typical Accuracy: Order of 10m –Accuracy of Ours: sub-meter –Refresh Rate: 10Hz –Drawback: Bulk –Result: Media Lab Courtyard

8. Hardware GPS Compass PalmPilot Laptop Headphones Microphone

9. Authoring and Playback Focus on user authoring –Most other systems do not –Simple! Playback is still very important –New author options lead to new playback issues

10. Authoring Author creates SoundSpots –In-Field –Using GUI

11. Playback: Basics User dons system Very simple start-up Moves through environment Hears sound tied to location

12. Playback: Spatialized Sound Each sound “appears” to be coming from center of a SoundSpot Java 3D API Compass attached to headphones Do not deal with environment effects (enclosed spaces, etc.) Future work will improve spatialized sound

13. Playback: Two Modes Explorer –All SoundSpots activated –Can be very cluttered –Ultimate control with listener Activation Networks

14. Playback: Activation Networks For Linking and Organizing Sounds Author creates Link Categories –e.g. My Family, About Media Lab Author associates Link Categories with a SoundSpot User can activate Link Category when standing in SoundSpot StartSpots

15. Playback: Activation Networks Example StartSpot in Boston’s Public Garden –Link Categories SMG: SMG SoundSpots Weeping Willow: interesting SoundSpots about weeping willow trees My Favorites: my favorite SoundSpots –When entering StartSpot, activate “Weeping Willow”; now those “exist” –Weeping Willow spots may have their own LinkCategories

16. Playback: Directing the User Palm Interface –Easy, intuitive –But uses visual information! Audio Beacons?

17. Security and Privacy Authentication for Authors Open System for Users Wireless Connection to Server?

18. Future Work Finish Implementation! More Portable System Temporal Information Other forms of audio augmentation

19. Contact and Contributors E-mail: jrozier@mit.edu http://smg.media.mit.edu Primary Contributors –Judith Donath –Karrie Karahalios –Birendro Roy and Tuan Phan