1 Damask A Tool for Early-Stage Design and Prototyping of Multi-Device User Interfaces G r o u p f o r User Interface Research University of California Berkeley James Lin UC Berkeley

2 Problem Design for Multiple Devices Computing world increasingly diverse Hard to design for variety of devices Use automatically generated UIs – results in UIs poorly optimized for device – only used as last resort by end-users

3 Problem Design for Multiple Devices Design each device-specific UI separately – rarely coordinated may lead to inconsistencies – tedious redesign every time new device comes out – time-consuming less time for prototyping

4 Solution Damask Tool for designing, prototyping multiple- device UIs – desktop web – cell phone – speech Uses design patterns to retarget one device-specific UI to other devices

5 Solution Damask Same quality Much less time – 1 device-specific UI + a little for each additional device Compared to designing each UI separately:

6 Outline Introduction Model-based UIs Design patterns Damask Research approach

7 Model-based UI Techniques Forces designer to start with abstract widgets – designers think in terms of concrete widgets Does not handle radically differently structured UIs well – use patterns to address this issue Abstract Model Device profile Specialized Application Model Translation Specialization Concrete Interface

8 Outline Introduction Model-based UIs Design patterns Damask Research approach

9 Patterns A general solution to a general problem, where the solution could be applied many times and never be the same twice Different levels of abstraction – site level: accessibility – storyboard level: shopping cart – page level: navigation bar  focus on latter two levels

10 Patterns Consist of 6 parts – name – background – problem – examples – solution – references

11 Patterns Examples Actual instances of patterns in practice

12 Patterns Solution Captures essence on how to solve pattern Navigation bar: First-level navigation Second-level navigation

13 Outline Introduction Model-based UIs Design patterns Damask Research approach – designing – retargeting – managing multi- device designs – running – creating patterns

14 Damask Three important elements – pattern explorer – pattern viewer – tabbed canvas: 1 tab/device

15 Damask Designing a User Interface Designer  sketches design – browses through patterns – merges device-specific pattern solution into design

16 Damask Designing a User Interface Designer – sketches design  browses through patterns – merges device-specific pattern solution into design Shopping cart PC solution

17 Damask Designing a User Interface Designer – sketches design – browses through patterns  merges device-specific pattern solution into design Shopping cart PC solution

18 Damask Designing a User Interface Designer – sketches design – browses through patterns  merges device-specific pattern solution into design Shopping cart

19 Damask Designing a User Interface Designer customizes pattern instance to fit particular project – this creates another example of the pattern – tool keeps track of customizations to pattern instance Shopping cart

20 Damask Retargeting a User Interface Designer picks another target device  tool instantiates pattern for target device, applies same customizations to this instance – results in generated UI for target device – can tweak generated UI Shopping cart Cell-phone solution

21 Damask Retargeting a User Interface Designer picks another target device – tool instantiates pattern for target device, applies same customizations to this instance  results in generated UI for target device – can tweak generated UI Shopping cart

22 Damask Retargeting a User Interface Designer picks another target device – tool instantiates pattern for target device, applies “same customizations” to this instance  results in generated UI for target device – can tweak generated UI Shopping cart

23 Damask Retargeting a User Interface Designer picks another target device – tool instantiates pattern for target device, applies same customizations to this instance – results in generated UI for target device  can tweak generated UI Shopping cart

24 Damask Retargeting a User Interface Designer picks another target device – tool instantiates pattern for target device, applies same customizations to this instance – results in generated UI for target device  can tweak generated UI Shopping cart

25 Damask Managing Multi-device Designs Changes within pages do not propagate – moving items – removing images Adding and removing pages do Emphasize structure over content Split canvas or create new window to see more than one device-specific UI

26 Damask Managing Multi-device Designs Mousing over design will highlight corresponding parts of other multi-device design

27 Damask Managing Multi-device Designs One desktop web page could correspond to several cell phone pages or vice-versa

28 Damask Managing Multi-device Designs Designer can specify where edits do not propagate

29 Damask Managing Multi-device Designs Designer can specify where edits do not propagate

30 Damask Running a User Interface Desktop and cell phone – generate HTML and WML or cHTML – simulate web browser internally Voice – generate SUEDE-compatible files – use SUEDE to run

31 Damask: Creating Patterns Make Pattern from Design Fragments 1. Select part of design Name: Shopping Cart Background Problem Examples Solution References Pattern: Shopping Cart Jump To: Solution

32 Damask: Creating Patterns Make Pattern from Design Fragments 2. Drag to Pattern sidebar Solution Desktop Cell Phone Pattern: Shopping Cart Jump To: Solution Shopping Cart

33 Damask: Creating Patterns Generalize Pattern Solution

34 Damask: Creating Patterns Generalize Pattern Solution

35 Damask: Creating Patterns Relate Device-Specific Solutions

36 Damask: Creating Patterns Relate Device-Specific Solutions

37 Outline Introduction Model-based UIs Design patterns Damask Research approach – interviews with cross- device UI designers – survey of design patterns – prototyping and building Damask – evaluation

38 Research Approach Interviews with Designers For a given multi-device app – all device-specific UIs designed at same time or different times – same group or different groups – how much communication if different groups – when was UI tested

39 Research Approach Interviews with Designers 4 designers, 1 developer at 3 companies – web portal, enterprise software, software 2 out of 3 companies – no team designed for both mobile and desktop – looked at desktop UI to get ideas for tasks, flow – few discussions with desktop designers, if any – used Visio to diagram flows 1 company – one person designed same interface for both PDA and desktop – but then separate groups for PDA, desktop (same manager)

40 Research Approach Survey of Design Patterns Examine multi-device web sites for patterns – e.g., Amazon, Expedia, Google, MSN – sketch out storyboards Read books, web sites on patterns – e.g., web design pattern book by Doug, James and Jason

41 Research Approach Prototyping and Building Damask Analyze findings from interviews and patterns survey Build lo-fi prototype – test with UI designers from industry Design architecture of Damask Build hi-fi prototype – use DENIM as initial code base

42 Research Approach Research Issues Applying customizations to patterns – which customizations to apply Consistency among device-specific UIs Showing relationships among device- specific UIs Handling multiple designers – especially tricky since design process in flux

43 Research Approach Evaluation Test – creating cross-device UI prototypes – creating and reusing patterns Approach – depends on results of interviews – possibilities design 2 apps, one with Damask, one without given design for app, retarget to another device

44 Research Approach Evaluation Metrics – speed – satisfaction with Damask – “quality” of designs judged by other designers – use of patterns frequency of use frequency of creation and reuse

46 Model-based UI Techniques “One of many list” could be rendered as – radio buttons – list – drop-down list Abstract Model Device profile Specialized Application Model Translation Specialization Concrete Interface

PC version Cell phone version

Voice example PC version Voice version Web Site Design View cartAdd to cart CartCart w/ new item Checkout Say acct no Acct numNo account Bump to operator Say PIN PIN Confirm YesNo Thanks! Back to square one

Canvas

Pattern Explorer

Pattern

Target device

56 Damask Managing Multi-device Designs Designer can specify where edits do not propagate

57 Damask Managing Multi-device Designs Designer can specify where edits do not propagate

61 Research Approach Architecture Issues Combine abstract and concrete UI models and design patterns – relationships not yet decided How are patterns embedded within UI models? Do the patterns themselves consist of models? – if so, abstract or concrete or both? How many models? – one model for UI that spans multiple devices – one model for each device-specific UI