2. Anywhere, Anytime, Anydevice Interfaces: Tools, Infrastructure, & Applications Summer 2002 BID/HCC Retreat for User Interface Research Group Prof. James Landay Katie Everitt, Wai-ling Ho-Ching, Jason Hong, Xiaodong Jiang, Hesham Kamel, Scott Klemmer, Jimmy Lin, Mark Newman, Anoop Sinha, Miriam Walker, Sarah Waterson University of California, Berkeley

3. 2 Design in the era of Ubicomp Future computing devices won’t have the same UI as current PCs wide range of devices small or embedded in environment often w/ “alternative” I/O & w/o screens information appliances lots of devices per user all working in concert How does one design for this environment? What will these interfaces look like? I-Land by Streitz, et. al.

4. 3 One Vision of Future User Interfaces Star Trek style UI verbally ask the computer for information may be common in mobile/hands-busy situations problem: hard to design, build, & use! need perfect speech recognition & language understanding

5. 4 Our Vision of Future Interfaces Multimodal, Context-aware UIs multimodal uses multiple input modalities (speech & gesture) to disambiguate user says “move it to this screen” while pointing context-aware apps aware of location, user, what they are doing, … people are talking -> don’t rely on speech I/O Problem: how to prototype & test new ideas?

6. 5 Outline of Updates Tools Designer’s Outpost DENIM SUEDE WebQuilt Design Patterns & Damask Infrastructure Context Fabric Applications Emergency Response

7. 6 Designers’ Outpost: A Tangible Interface for Designing Info Architectures Combines the physical & virtual physical post-its, virtual feedback Supports existing practice affordances of paper collaboration large, persistent representation Adds advantages of e-media editing, reuse, distribution, hand-off Has an informal user interface only recognizes where notes are Status finished design history adding support for remote collab

8. 7 DENIM: Designing Web Sites by Sketching DENIM supports web design practice from initial information architecture to testing integrated multiple views informal sketching UI DENIM adds to practice lo-fi interactive prototypes advantages of e-media Status downloaded over 7000 times looking to add staff programming support to make this a truly useful tool & do longitudinal studies of use

9. 8 SUEDE: Informal Prototyping for Speech-based UIs SUEDE supports speech- based UI design moving from concrete examples to abstractions embeds iterative design fast, fluid, informal interface designers need not be speech technology experts Status downloaded over 1000 times making usable for real projects (e.g., cut, copy, paste)

10. 9 WebQuilt: Remote Web Usability Testing No observer present for test more realistic & easier to scale up Easy to do testing early Collect clickstream data path through a website task-based use proxy to record WebQuilt analysis tool for usability experts & designers quick, easy, understandable allow data exploration Status new visualization implemented on-going work expands to WAP & other mobile devices

11. Interface

12. Overview

13. Storyboard view

15. Page view

16. 15 Design Patterns Design is about finding solutions unfortunately, designers often reinvent hard to know how things were done before & to reuse solutions design patterns allow designers to reuse what works well First used in architecture [Alexander] Communicate design problems & solutions how to create a beer garden where people socialize… how big doors should be & where… how to use handles… Not too general & not too specific use solution “a million times over, without ever doing it the same way twice”

17. 16 Web Design Patterns We’ve written a new book on this topic! van Dunye, Landay, & Hong (available July 2002) Communicate web design problems & solutions how to make thriving online communities… how to create a shopping cart that supports check out… how to create navigation bars for finding relevant content…

18. 17 Pattern Examples Navigation bar

19. 18 Pattern Examples Navigation bar

20. 19 Pattern Solution Captures essence on how to solve problem Navigation bar Generality of solution fits informal approach! First-level navigation Second-level navigation

21. 20 Damask: Using Patterns for Cross Device UI Design Designer sketches design browses through patterns merges device-specific pattern generalization into design

22. 21 Damask: Using Patterns for Cross Device UI Design Designer sketches design browses through patterns merges device-specific pattern generalization into design Shopping cart PC version

23. 22 Damask: Using Patterns for Cross Device UI Design Designer sketches design browses through patterns merges device-specific pattern generalization into design

24. 23 Damask: Using Patterns for Cross Device UI Design Designer customizes solution to fit project this creates another example of the pattern tool keeps track of transformations

25. 24 Damask: Using Patterns for Cross Device UI Design Designer picks another target device tool takes pattern generalization for target device, applies same transformations results in generated UI for target device designer continues to fix & customize result Shopping cart Cell-phone version

26. 25 Examples of Using Context Context TypesExisting ExamplesHuman Concern Room ActivityAuto Lights On / OffConvenience Personal Identity & Time File SystemsFinding Info TimeCalendar RemindersMemory Activity Finding Info Safety Time Location Activity Health Alert Tag Photos History Identity Proximity Efficiency Service Fleet Dispatching Context Types Potential ExamplesHuman Concern

27. 26 Emergency Response Scenario Fire or earthquake situation Keep track of the people in a building allow building managers to check if a building is clear in the event of an evacuation allow firefighters to check where people were Provide reasonable privacy protection people don't like to be tracked emergency situations relatively rare

28. 27 Building InfoSpace Carol's InfoSpace Smart Dust User="Carol" Location="525 Soda Hall" Time="Apr 12 1:05PM" Access Control User="Carol" Location="5th floor" Age="37 seconds" Send location info Logging User="Carol" Location="in" Age="37 seconds" Blurring Notification Emergency Response Scenario

29. 28 Context Fabric: Scalability & Privacy for Context-Aware Computing Provide network-oriented set of abstractions, mechanisms, & programming model Scalability data-oriented repositories – information spaces decentralized, high availability, with local control Privacy provide suite of mechanisms for app developers based on Fair Information Practices & Information Asymmetry Status implementing architecture & initial apps

30. Anywhere, Anytime, Anydevice Interfaces: Tools, Infrastructure, & Applications Summer 2002 BID/HCC Retreat for User Interface Research Group Prof. James Landay Katie Everitt, Wai-ling Ho-Ching, Jason Hong, Xiaodong Jiang, Hesham Kamel, Scott Klemmer, Jimmy Lin, Mark Newman, Anoop Sinha, Miriam Walker, Sarah Waterson University of California, Berkeley