1. End-User Perceptions of Formal and Informal Representations of Web Sites Jason Hong Francis Li James Lin James Landay Group for User Interface Research UC Berkeley CHI 2001 April 5, 2001

2. Design Representations Differing levels of detail Low-fidelity (Lo-fi) High-fidelity (Hi-fi)

3. Static Images Hi-fi representations – encourage discussion of details fonts colors graphics pixel-level layout Lo-fi representations – encourage discussion of high- level issues information architecture navigation flow overall page layout

4. Interactive Prototypes Hi-fi prototypes – use hi-fi representations Lo-fi prototypes – use lo-fi representations – often on paper – faster to create than hi-fi prototypes

5. End-User Perceptions How would end-users judge a web site design, depending on the fidelity of the site’s prototype?

6. Hypotheses Between hi-fi and lo-fi prototypes – people will find many of the same usability problems

7. Hypotheses (cont’d) Between hi-fi and lo-fi prototypes – people will judge the following attributes differently: professional functional finished likely to change

8. Hypotheses (cont’d) Between hi-fi and lo-fi prototypes – people will rank the importance of visual vs. non- visual design issues differently visual issues: font, color, graphics non-visual issues: general location of navigation bars, overall page layout, organization of information

9. Experiment Design Each participant evaluated one site with hi-fi, other site with lo-fi Within-subjects Hi-fiLo-fiTotal Site A448 Site B448 Total8816 Between- subjects

10. Experiment Design For hi-fi prototypes – downloaded two actual web sites – pared down product list – greeked some text out – hardwired shopping cart data

12. Experiment Design For lo-fi prototypes – made electronic representations in DENIM instead of using paper – exported DENIM pages to GIF + image maps – touched up sketches to improve handwriting – added HTML text fields by hand

16. Experiment Design Participants asked to do 5 tasks per web site – find a specific piece of information – find another specific piece of information – find information deeper in site – add two particular items to shopping cart – purchase items in shopping cart

17. Experiment Design After each web site, participants: – gave overall impressions of site design (scale: 0–10) professional, finished, functional, likely to change – ranked ten possible ways of improving web site 6 were non-visual improvements 4 were visual improvements – made comments, suggested improvements to web site

18. Significant differences found (p <.05, 2 tailed independent t-test)  Evaluated each site separately – between-subjects analysis Are the two web sites the same? – between hi-fi sites attractiveness quality security likelihood of changing – between lo-fi sites ease of use efficiency complexity attractiveness usefulness

19. Usability Problems Many of the same usability problems were found in both hi-fi and lo-fi – e.g., Add to Shopping Cart / Registration problem Lo-fi prototypes – sufficient for task-based usability evaluation – faster and easier to create

20. Perceptions of Design Attributes People did judge attributes differently depending on fidelity Lo-fi representations look: – less professional – less functional – less finished – more likely to change

21. Rankings of Visual and Non-visual Issues  Importance of visual vs. non-visual issues did not correlate with fidelity of prototype – electronic medium may raise expectations – quality of sketches may affect perception

22. Future Work Study earlier stages of design cycle Task-based vs. non-task-based evaluation Investigate effect of medium on perception – paper sketches – scanned paper sketches – electronic sketches – printed electronic sketches

23. Conclusions Many of the same usability problems found in hi-fi and lo-fi representations Lo-fi representations look: – less professional – less functional – less finished – more likely to change No correlation between importance of visual vs. non- visual improvements and fidelity of representation

24. For More Information http://guir.berkeley.edu/pubs/chi2001/formal- informal.pdf

25. Thanks!

26. Web Site Design Process

27. Experiment Design After each task, participants were asked how well site supported task (scale: 0–10) – easy – efficient – clear – helpful

28. More descriptive text in links6 Improved navigation bars4 Clearer indicators of what can be clicked on 9 Clearer scheme for organizing pages on the web site 8 Better page layout5 More streamlined shopping cart and checkout 10 More emphasis on web site brand7 Better use of fonts2 Better use of colors1 Better use of graphics3 Visual Non-Visual

29. More descriptive text in links6 5.5 Improved navigation bars4 5.5 Clearer indicators of what can be clicked on 9 9 Clearer scheme for organizing pages on the web site 8 9 Better page layout5 5.5 More streamlined shopping cart and checkout 10 9 More emphasis on web site brand7 5.5 Better use of fonts2 2 Better use of colors1 2 Better use of graphics3 2 Visual Non-Visual

30. More descriptive text in links6 5.5 Improved navigation bars4 2 Clearer indicators of what can be clicked on 9 9 Clearer scheme for organizing pages on the web site 8 9 Better page layout5 5.5 More streamlined shopping cart and checkout 10 9 More emphasis on web site brand7 5.5 Better use of fonts2 2 Better use of colors1 2 Better use of graphics3 2 Visual Non-Visual 7.25 2.875

31. ParticipantInformalFormal Non-VisualVisualNon-VisualVisual 17.252.875 2 3 4 5 Mean

32. ParticipantInformalFormal Non-VisualVisualNon-VisualVisual 17.252.8756.6673.75 2 3 4 5 Mean

33. ParticipantInformalFormal Non-VisualVisualNon-VisualVisual 17.252.8756.6673.75 26.0834.6256.0834.625 37.252.8756.0834.625 45.5 3.758.125 53.758.1254.9176.375 Mean

34. ParticipantInformalFormal Non-VisualVisualNon-VisualVisual 17.252.8756.6673.75 26.0834.6256.0834.625 37.252.8756.0834.625 45.5 3.758.125 53.758.1254.9176.375 Mean5.9674.85.5