1. Beyond the PC Kiosks & Handhelds Albert Huang Larry Rudolph Oxygen Research Group MIT CSAIL

2. Overview  Computation is useful everywhere  We need to explore interaction modes beyond the desk:  Kiosks  Handhelds  These are mutually complimentary

3. OK-Net Kiosk

4. Kiosk Specifications  Touch screen monitor  Small computer inside kiosk  Minimal infrastructure  Hacker-hardened  Nothing exposed

5. Under the hood

6. Self contained

7. Interaction Modes  General Public  Similar to web browser: point-and-click  Content harvested automatically  seminars, events, directory, news, etc.  Adapt to user  extension of user’s digital world  extension of user’s mobile devices

8. Identify User by Doodle  People initially interact with a doodle  Conjecture: doodles are unique  like signature  Provides a moderate amount of id  strong id not necessary

9. Other Interactions  User input  touch & speech  phone and PDA as remote finger  Information Transfer  SMS  email  Bluetooth (OBEX push)

10. Simple Kiosk App: Stata Guidance  Stata is confusing  Kiosk provides several guide modes  Passive:  show & push map to Bluetooth-enabled device  Active:  guide user along the way

11. A graph for each floor Nodes: junctions or destinations

12. Given start and destination compute a path

13. Send it all to phone

14. Real-time navigation  Track user (phone) in Stata  Trivial deployment  Bluetooth beacon in each PC  < $20 per beacon  Indoor GPS for phones  scan for Bluetooth beacons  map detected beacons to a location

15. Real-time navigation

20. Improvements needed  Takes too long to recognize beacons  Much better results with two beacons  Signal comes and goes  Incorporate model of human motion  Probabilistic filtering

21. Some people cannot read maps  A “human-centric” navigation guide  without sound  without abstraction  A picture is worth 1000 words  photo sequence

27. Digital Assistant  Kiosks can send content to phones  But users also generate content  pictures  audio memos  text memos  selections

28. Human-centric organization  Organize as it happens  Associate content for subsequent retrieval  Display associated content together  Content can be displayed manually or automatically

29. External Triggers  Time (the usual alarms)  Location  enter super market, home, car, office  Meeting someone  Phone call from someone  Many others to be discovered

30. Testbed: Conference Domain  When attending technical conference  Select interesting talks at kiosk  Send details to phone  Take notes during a talk  camera for photos, videos, printed text  microphone for audio notes, annotations  keypad, laptop for text notes  Associate content with events

31. Process content offline  Apply recognition technologies for  images  OCR, Object, People, Place  audio  speech transcription  video  sketch & gesture  Build semantic web to integrate with services  Potential use  Automatically generate conference report

32. Sharing content  Content tagged as public or private  Phone-phone or phone-kiosk interaction synchronizes public content  Content spreads throughout community  each new recipient tags it anew  More complicated than public/private  User may want to share content among her own devices, e.g. laptop, phone, iPod, etc.  User may want to share some content with her boyfriend’s devices

33. Conclusion about Kiosks  Kiosks: new interaction model?  are they just glorified web browsers?  interaction with hand-held devices  proximity provides simple, physical interface