1. A centre of expertise in digital information managementwww.ukoln.ac.uk Benchmarking Your Web Site Brian Kelly UKOLN University of Bath Bath, BA2 7AY Email B.Kelly@ukoln.ac.uk URL http://www.ukoln.ac.uk/ UKOLN is supported by:

2. A centre of expertise in digital information managementwww.ukoln.ac.uk 2 Contents Introduction Why Benchmark? What Is Benchmarking? Tools For Testing Web Sites From Testing To Benchmarking Case Studies: WebWatch Surveys Of UK University Sector Commercial Approaches Discussion

3. A centre of expertise in digital information managementwww.ukoln.ac.uk 3 What Do You Think Benchmarking is? Could you turn to your neighbour and ask: What do you think benchmarking is? How you think benchmarking can help you?

4. A centre of expertise in digital information managementwww.ukoln.ac.uk 4 Benchmarking: A Definition Benchmarking is about identifying and measuring best practice processes that work elsewhere and then emulating them. The aim is to reduce duplication by learning from others who have already found the solution. It is about: Understanding your weaknesses Comparison with your peers Note that best practices are constantly evolving.

5. A centre of expertise in digital information managementwww.ukoln.ac.uk 5 Aims Of This Talk By the end of the session you should: Be able to benchmark your Web site in relation to other sites in your community Have seen examples of use of auditing and evaluating tools Have considered other types of benchmarking activity available Be in a position to decide whether to adopt this methodology in your organisation

6. A centre of expertise in digital information managementwww.ukoln.ac.uk 6 Approaches To Benchmarking There are a number of approaches to benchmarking of Web sites: Manual Benchmarking Use of manual techniques such as questionnaires, usability studies, etc. Automated Benchmarking Automated benchmarking makes use of software tools to support benchmarking Approaches can include: Dedicated benchmarking software products, typically running on desktop PC Use of benchmarking services available on the Web

7. A centre of expertise in digital information managementwww.ukoln.ac.uk 7 Benchmarking Approaches Manual: End users involved in process Can receive feedback which cannot be obtained using software Can be time-consuming and expensive Automated: Can be less expensive Can scale to many thousands of resources Restricted to aspects which software can process This talk deals mainly with automated benchmarking, using Web-based tools

8. A centre of expertise in digital information managementwww.ukoln.ac.uk 8 Web Testing Services Many Web-based services are available for reporting on various aspects of Web sites, including: HTML compliance Browser compatibility Broken Links Load Time Accessibility Link popularity …

9. A centre of expertise in digital information managementwww.ukoln.ac.uk 9 Example - NetMechanic NetMechanic: A Web-based service for checking Web sites Various functions available for free Additional functions such as more comprehensive testing are licensed http://www.netmechanic.com/

10. A centre of expertise in digital information managementwww.ukoln.ac.uk 10 Example – Doctor HTML Doctor HTML: A Web page analysis tool Free for testing individual pages A licensed version can be installed locally for checking entire Web sites http://www2.imagiware.com/RxHTML/

11. A centre of expertise in digital information managementwww.ukoln.ac.uk 11 Example - Bobby Bobby: A Web-based accessibility checker Can test individual pages A licensed downloadable version can check entire Web sites http://bobby.watchfire.com/

12. A centre of expertise in digital information managementwww.ukoln.ac.uk 12 Example – Link Popularity Link popularity: An indication of the popularity of a Web site Can be obtained by analysing search engines such as AltaVista and Google http://www.linkpopularity.com/

13. A centre of expertise in digital information managementwww.ukoln.ac.uk 13 Example – Server Analysis Netcrafts server analysis provides details of: Web server software Operating system environment Server availability (limited) Nos. of servers in domain http://www.netcraft.com/

14. A centre of expertise in digital information managementwww.ukoln.ac.uk 14 Example – Server Analysis The University of Dundee provide a HTTP analysis tool: Analyses HTTP headers http://www.somis.dundee.ac.uk/ general/wizards/fetchhead.html http://www.somis.dundee.ac.uk/ general/wizards/fetchhead.html

15. A centre of expertise in digital information managementwww.ukoln.ac.uk 15 Usability Usability normally requires manual testing However automated support tools are also available such as WebSAT http://zing.ncsl.nist.gov/ WebTools/WebSAT/operation.html http://zing.ncsl.nist.gov/ WebTools/WebSAT/operation.html

16. A centre of expertise in digital information managementwww.ukoln.ac.uk 16 From Testing To Benchmarking These tools are typically used for testing individual pages on ones own Web site However applying tools across other Web sites allows: Comparisons to be made with competitors and collaborators Examples of best practices to be found and lessons learnt from Examples of problems to be found and mistakes avoided Trends to be monitored by repeat surveys Limitations of tools to be found by testing across wide range of Web site environments

17. A centre of expertise in digital information managementwww.ukoln.ac.uk 17 UK HE Case Studies Every 3 months a WebWatch survey is published in the Ariadne e-journal: Surveys include: Accessibility of entry points Nos. of Web servers Nos. of links to organisation Size of entry points Web server software Relationships Together with manual surveys of: Search engine software 404 error pages See: Case Studies

18. A centre of expertise in digital information managementwww.ukoln.ac.uk 18 Accessibility In September 2002 Bobby was used to analyse the entry points the UK University Web sites. http://www.ariadne.ac.uk/issue33/web-watch/ Findings Only 4 pages appeared to comply with Bobby AA guidelines Further analysis revealed that only 3 complied Case Studies

19. A centre of expertise in digital information managementwww.ukoln.ac.uk 19 Size Of Home Page The size of UK University entry points was analysed in 1998 and repeated in June 2001 http://www.ariadne.ac.uk/issue28/web-watch/ The reasons for the large entry points was reviewed. Case Studies

20. A centre of expertise in digital information managementwww.ukoln.ac.uk 20 Numbers Of Web Servers A survey of the numbers of Web servers was carried out in 2000 and repeated in 2002 http://www.ariadne.ac.uk/issue31/web-watch/ Most institutions have a small number of Web servers but a few have over 100 Case Studies

21. A centre of expertise in digital information managementwww.ukoln.ac.uk 21 Whats Related? Netscapes Whats Related tool was used to record: Popularity Nos. of pages indexed Nos. of links to site http://www.ariadne.ac.uk/ issue27/web-watch/

22. A centre of expertise in digital information managementwww.ukoln.ac.uk 22 Links To University Web Sites A survey of the number of links to UK University Web sites was published in 2000 The survey used the AltaVista and Infoseek search engines http://www.ariadne.ac.uk/issue23/web-watch/

23. A centre of expertise in digital information managementwww.ukoln.ac.uk 23 Search Engines A manual survey of search engines used on UK University Web sites was carried out in 1997 and has been repeated every 6 months in order to monitor trends http://www.ariadne.ac.uk/issue30/web-watch/

24. A centre of expertise in digital information managementwww.ukoln.ac.uk 24 404 Error Pages A (manual ) survey of 404 error pages on UK University Web sites was carried out in 1999 and repeated in 2002. WebWatch: 404s - What's Missing? June 1999 Revisiting 404 Error Pages In UK University Web Sites, June 2002, WebWatch: 404s - What's Missing? June 1999 Revisiting 404 Error Pages In UK University Web Sites, June 2002, The original survey and article helped to raise the importance of well- designed 404 pages as an important navigation feature Case Studies

25. A centre of expertise in digital information managementwww.ukoln.ac.uk 25 404 Error Pages Significant changes have been made since the findings of the first survey was published Apache/1.3.6 Server at www.shef.ac.uk Port 80

26. A centre of expertise in digital information managementwww.ukoln.ac.uk 26 Limitations Of This Approach What limitations do you think this approach may have? Limitations

27. A centre of expertise in digital information managementwww.ukoln.ac.uk 27 Limitations Reliance on third party tools Inconsistencies across tools Unusual aspects of Web sites Inadequacies of automated tools Cant handle Intranets Performance implications Legal and ethical issues Personalision, cookies, etc. Limitations

28. A centre of expertise in digital information managementwww.ukoln.ac.uk 28 Reliance On Third Party Tools This approach relies on use of third party software: Company may go out of business Company may introduce charging or change conditions of use Companies may change format of its output Company may change algorithms (possibly without notification) Example: The Bobby accessibility checker withdrew a summary of the file size of resources. CAST sold Bobby: the new company introduced limitations to use of Bobby Example: The Bobby accessibility checker withdrew a summary of the file size of resources. CAST sold Bobby: the new company introduced limitations to use of Bobby Limitations

29. A centre of expertise in digital information managementwww.ukoln.ac.uk 29 Intranets, etc. Use of public third party Web sites for testing: May not work with Intranets or Web sites which require a username and password to access Possible Solution Some testing services allow you to give a username and password If you do this, you are trusting the service not to steal the username and password! Limitations

30. A centre of expertise in digital information managementwww.ukoln.ac.uk 30 Inconsistencies Different tools may have inconsistent approaches Limitations Example NetMechanic and Bobby (previous version) reported on the file size of analysed pages. Bobby only analysed the HTML page and inline images. NetMechanic also included external JavaScript and stylesheet files. NetMechanic respected the Standard for Robot Exclusion (SRE) and would not analyse images if the SRE banned access. Bobby ignored the SRE. Example NetMechanic and Bobby (previous version) reported on the file size of analysed pages. Bobby only analysed the HTML page and inline images. NetMechanic also included external JavaScript and stylesheet files. NetMechanic respected the Standard for Robot Exclusion (SRE) and would not analyse images if the SRE banned access. Bobby ignored the SRE.

31. A centre of expertise in digital information managementwww.ukoln.ac.uk 31 Inadequacies Automated tools: Are not suitable for testing all aspects of a Web site Need to be complemented by manual testing Reliance on automated results without warning notices can cause confusion Limitations Accessibility Testing Automated tools such as Bobby can report on missing ALT tags However automated tools cannot report that a meaningful ALT tag is given alt=>

32. A centre of expertise in digital information managementwww.ukoln.ac.uk 32 Performance Implications Automated tools: Could degrade the performance of Web sites if poorly designed Limitations Case Study A HTML validation tool was used to check A Web site. Shortly after it was used, it was found that it repeatedly sent HTTP requests to a Web site, which felt that this was a denial-of-service attack. Case Study A HTML validation tool was used to check A Web site. Shortly after it was used, it was found that it repeatedly sent HTTP requests to a Web site, which felt that this was a denial-of-service attack.

33. A centre of expertise in digital information managementwww.ukoln.ac.uk 33 Legal And Ethical Issues If your survey findings: Give a negative impression of a Web site Are flawed, and give an mistaken negative impression of a Web site could you be sued? Limitations Case Study WebWatch surveys seek to highlight examples of best practices. Care is taken in the language used when problems are reported. Case Study WebWatch surveys seek to highlight examples of best practices. Care is taken in the language used when problems are reported.

34. A centre of expertise in digital information managementwww.ukoln.ac.uk 34 Personalisation How should testing tools behave if Web sites provide personalised interfaces: Make use of username details to personalise context Personalise the interface based on the users browser Personalise the interface based on other environment factors (e.g. time, referer page, language setting, etc.) Limitations

35. A centre of expertise in digital information managementwww.ukoln.ac.uk 35 Unusual Aspects How should testing tools deal with other unusual aspects of a Web site such as: Web page redirects Splash screens Pop-up windows Frames etc. Limitations Example When you give a URL the Web page is redirected to another URL. The new page displays a splash screen for 5 seconds and then moves to a new page with contains frames. In addition a pop-up window is displayed. How many pages are there? Example When you give a URL the Web page is redirected to another URL. The new page displays a splash screen for 5 seconds and then moves to a new page with contains frames. In addition a pop-up window is displayed. How many pages are there?

36. A centre of expertise in digital information managementwww.ukoln.ac.uk 36 Benchmarking And QA The benchmarking approach may be used to: Ensure that Web sites comply with standards and best practices May be of interest to funding bodies UKOLN involved in work in this area to ensure that projects comply with standards and best practices in order that they will be interoperable See < http://www.ukoln.ac.uk/ qa-focus/surveys/ > See < http://www.ukoln.ac.uk/ qa-focus/surveys/ >

37. A centre of expertise in digital information managementwww.ukoln.ac.uk 37 Who Else Is Doing This? Who else may be carrying out benchmarking surveys? We will use a Google search for: accessibility surveys, Web site benchmark,HTML compliance surveys, etc. In order to explore other approaches including: Commercial approaches Non-commercial approaches

38. A centre of expertise in digital information managementwww.ukoln.ac.uk 38 US Accessibility Surveys Axel Schmetzke has carried out surveys of the accessibility of selected US University Web sites http://library.uwsp.edu/aschmetz/ Accessible/websurveys.htm http://library.uwsp.edu/aschmetz/ Accessible/websurveys.htm

39. A centre of expertise in digital information managementwww.ukoln.ac.uk 39 Try It For Yourself The methodology which has been described can be used by yourself across your community Benefits: You will get an idea of how your compare with your peers For national bodies, funders, etc. you can gain a profile of your community There may be opportunities for describing your community at conferences, etc.

40. A centre of expertise in digital information managementwww.ukoln.ac.uk 40 Implementing A Benchmark Survey To implement your own benchmark across a community you can simply examine WebWatch articles and adapt the HTML for your own use. Further details at http://bobby.cast.org/bobby/bobbyServlet? URL=http%3A%2F%2Fwww2.brent.gov.uk%2F&output=Submit&gl=wcag1-aaa Technique Used Use the Web service on a site Copy URL into template Determine URL structure Use as basis for use with other URLs

41. A centre of expertise in digital information managementwww.ukoln.ac.uk 41 Implementation (2) Simple technique: Copy URLs into a template Better technique: Create HTML file using a server-side script Better technique: Use a backend database so resources can be more easily managed Try it … Do for all websites Try it …

42. A centre of expertise in digital information managementwww.ukoln.ac.uk 42 Next Generation Tools We can expect to see further development in testing tools: Why? Compliance with, say, e-Government guidelines Ensure Web sites work e.g. e-commerce How: Tools which provide richer functionality (e.g. dealing with personalised Web sites) Development of Web services for testing Agreement on standards e.g. what is a Web page Development of XML standards for interchange of results e.g. EARL

43. A centre of expertise in digital information managementwww.ukoln.ac.uk 43 Resources For You To Use A series of exercises on Web site benchmarking is available, which contains details of a number of benchmarking tools http://www.ukoln.ac.uk/web-focus/events/ conferences/ucisa-tlig-2002/benchmarking/

44. A centre of expertise in digital information managementwww.ukoln.ac.uk 44 Questions Any questions?