Content or Graphics? An Empirical Analysis of Criteria for Award-Winning Websites Rashmi Sinha, Marti Hearst & Melody Ivory University of California, Berkeley Maya Draisin International Academy of Digital Arts and Sciences For more information: http://sims.berkeley.edu/~sinha/Webby2000 Content or Graphics

What are the qualities of award-winning websites Webby Awards dataset is an untapped resource Large pool of sites are rated on multiple criteria Analysis of the dataset can inform us about the qualities that distinguish good from bad websites Content or Graphics 7th Conference on Human Factors and the Web

Ultimate Goal: Tools to Help Non-Professional Designers Examples: A “grammar checker” to assess guideline conformance Imperfect Only suggestions – not dogma Automatic comparison to highly usable pages/sites Automatic template suggestions Content or Graphics 7th Conference on Human Factors and the Web

The Current Goal What are the qualities of award-winning websites? Do good websites in different content categories differ other in terms of importance of different criteria? What is more important for website quality: Content or Visual Design Content or Graphics 7th Conference on Human Factors and the Web

Webby Awards 2000 Fifth Year in operation Webby Awards 2000: 3000 websites from 27 content categories rated on multiple criteria Content or Graphics 7th Conference on Human Factors and the Web

Webby Awards 2000 6 criteria Scale: 1-10 (highest) Overall Site Experience Five specific Criteria Content Structure and Navigation Visual Design Interactivity Functionality Scale: 1-10 (highest) Content or Graphics 7th Conference on Human Factors and the Web

Judging Process Open Submission Process 3 rounds of Judging Anyone can submit a site 3 rounds of Judging Site Review Stage (from 3000 to 400 sites) Site Nominating Stage (from 400 to 135 sites) Final Judging (from 135 to 27) Content or Graphics 7th Conference on Human Factors and the Web

Webby Judges Review Stage Judges Nominating Judges Internet professionals. Familiarity with content category Nominating Judges New media journalists, editors, web developers, and other Internet professionals Possess comprehensive knowledge of sites in their content category People’s Voice Ratings People vote for their favorite sites of the final 135 Content or Graphics 7th Conference on Human Factors and the Web

Review Stage Analysis 6 criteria 3000 sites 3 judges rated each site Content, Structure & Navigation, Visual Design, Functionality & Interactivity Overall experience 3000 sites 3 judges rated each site Content or Graphics 7th Conference on Human Factors and the Web

The whole range of sites: Good to Bad Mean = 6.01 SD = 1.59 Content or Graphics 7th Conference on Human Factors and the Web

What criteria contribute to overall rating? Content or Graphics 7th Conference on Human Factors and the Web

What factors predict ratings The best predictor of the overall score is content. Also the best unique predictor. The worst unique predictor is visual design Content or Graphics 7th Conference on Human Factors and the Web

Review Stage: Interrelationship of Criteria Content or Graphics 7th Conference on Human Factors and the Web

Nominating Stage Analysis 6 criteria Content, Structure & Navigation, Visual Design, Functionality & Interactivity Overall experience 414 sites 3 judges rated each site Content or Graphics 7th Conference on Human Factors and the Web

Mostly Higher Rated Sites Mean = 7.6 SD = 1.66 Content or Graphics 7th Conference on Human Factors and the Web

What criteria contribute to overall rating? Content or Graphics 7th Conference on Human Factors and the Web

Example Webby Content Categories Art, Commerce & Radio Content or Graphics 7th Conference on Human Factors and the Web

Art Content or Graphics 7th Conference on Human Factors and the Web

Commerce Sites Content or Graphics 7th Conference on Human Factors and the Web

Radio Sites Content or Graphics 7th Conference on Human Factors and the Web

Conclusions Content is by far the best predictor of overall site experience. Visual Design does not have much predictive power Content or Graphics 7th Conference on Human Factors and the Web

More information: http://sims.berkeley.edu/~sinha/Webby2000 Content or Graphics 7th Conference on Human Factors and the Web