1. Content or Graphics? An Empirical Analysis of Criteria for Award-Winning Websites
Rashmi Sinha, Marti Hearst & Melody Ivory
University of California, Berkeley
Maya Draisin
International Academy of Digital Arts and Sciences
For more information:
Content or Graphics

2. What are the qualities of award-winning websites
Webby Awards dataset is an untapped resource
Large pool of sites are rated on multiple criteria
Analysis of the dataset can inform us about the qualities that distinguish good from bad websites
Content or Graphics
7th Conference on
Human Factors and the Web

3. Ultimate Goal: Tools to Help Non-Professional Designers
Examples:
A “grammar checker” to assess guideline conformance
Imperfect
Only suggestions – not dogma
Automatic comparison to highly usable pages/sites
Automatic template suggestions
Content or Graphics
7th Conference on
Human Factors and the Web

4. The Current Goal What are the qualities of award-winning websites?
Do good websites in different content categories differ other in terms of importance of different criteria?
What is more important for website quality: Content or Visual Design
Content or Graphics
7th Conference on
Human Factors and the Web

5. Webby Awards 2000 Fifth Year in operation Webby Awards 2000:
3000 websites from 27 content categories rated on multiple criteria
Content or Graphics
7th Conference on
Human Factors and the Web

6. Webby Awards 2000 6 criteria Scale: 1-10 (highest)
Overall Site Experience
Five specific Criteria
Content
Structure and Navigation
Visual Design
Interactivity
Functionality
Scale: 1-10 (highest)
Content or Graphics
7th Conference on
Human Factors and the Web

7. Judging Process Open Submission Process 3 rounds of Judging
Anyone can submit a site
3 rounds of Judging
Site Review Stage (from 3000 to 400 sites)
Site Nominating Stage (from 400 to 135 sites)
Final Judging (from 135 to 27)
Content or Graphics
7th Conference on
Human Factors and the Web

8. Webby Judges Review Stage Judges Nominating Judges
Internet professionals.
Familiarity with content category
Nominating Judges
New media journalists, editors, web developers, and other Internet professionals
Possess comprehensive knowledge of sites in their content category
People’s Voice Ratings
People vote for their favorite sites of the final 135
Content or Graphics
7th Conference on
Human Factors and the Web

9. Review Stage Analysis 6 criteria 3000 sites 3 judges rated each site
Content, Structure & Navigation, Visual Design, Functionality & Interactivity
Overall experience
3000 sites
3 judges rated each site
Content or Graphics
7th Conference on
Human Factors and the Web

10. The whole range of sites: Good to Bad
Mean = 6.01
SD = 1.59
Content or Graphics
7th Conference on
Human Factors and the Web

11. What criteria contribute to overall rating?
Content or Graphics
7th Conference on
Human Factors and the Web

12. What factors predict ratings
The best predictor of the overall score is content. Also the best unique predictor.
The worst unique predictor is visual design
Content or Graphics
7th Conference on
Human Factors and the Web

13. Review Stage: Interrelationship of Criteria
Content or Graphics
7th Conference on
Human Factors and the Web

14. Nominating Stage Analysis
6 criteria
Content, Structure & Navigation, Visual Design, Functionality & Interactivity
Overall experience
414 sites
3 judges rated each site
Content or Graphics
7th Conference on
Human Factors and the Web

15. Mostly Higher Rated Sites
Mean = 7.6
SD = 1.66
Content or Graphics
7th Conference on
Human Factors and the Web

16. What criteria contribute to overall rating?
Content or Graphics
7th Conference on
Human Factors and the Web

17. Example Webby Content Categories Art, Commerce & Radio
Content or Graphics
7th Conference on
Human Factors and the Web

18. Art
Content or Graphics
7th Conference on
Human Factors and the Web

19. Commerce Sites Content or Graphics 7th Conference on
Human Factors and the Web

20. Radio Sites Content or Graphics 7th Conference on
Human Factors and the Web

21. Conclusions
Content is by far the best predictor of overall site experience.
Visual Design does not have much predictive power
Content or Graphics
7th Conference on
Human Factors and the Web

22. More information: http://sims.berkeley.edu/~sinha/Webby2000
Content or Graphics
7th Conference on
Human Factors and the Web