1. A centre of expertise in digital information managementwww.ukoln.ac.uk Making Effective Use Of Benchmarking Tools Brian Kelly UKOLN University of Bath Bath Email: B.Kelly@ukoln.ac.uk UKOLN is supported by: http://www.ukoln.ac.uk/web-focus/events/workshops/swmlac-2005-10/ This presentation describes the advantages and disadvantages of benchmarking tools and ways of maximising the effectiveness of such tools.

2. A centre of expertise in digital information managementwww.ukoln.ac.uk 2 Background Problems You've identified some problem areas for users of Web sites: Functionality – it doesn't work Usability – it's difficult for me to use Accessibility – it's difficult for people with disabilities to use Solutions Let's review the approaches you've taken to spot such problems and explore how to use the tools effectively: Tools that can help Processes that can help A Quality Assurance (QA) framework

3. A centre of expertise in digital information managementwww.ukoln.ac.uk 3 Useful tools: W3C's HTML validator: can spot functionality & accessibility problems Tools: Functionality (1) HTML, CSS, … Validation: Web page doesn't look right in my browser First thing: validate page! Tools W3C's CSS validator RSS validator (if you have an RSS newsfeed) …

4. A centre of expertise in digital information managementwww.ukoln.ac.uk 4 Tools: Functionality (2) Link Checking: Clear need to ensure links work Many tools available Validated part of my Web area Findings: 12,514 Web pages! Only checked internal links Large no. of errors – but vast majority false errors Some errors found in areas provided by others Others my fault – and mostly fixed Issues: We can't always rely on tools Why weren't errors spotted previously? What to do with large no. of errors? Issues: We can't always rely on tools Why weren't errors spotted previously? What to do with large no. of errors?

5. A centre of expertise in digital information managementwww.ukoln.ac.uk 5 http://webxact.watchfire.com/ Tools: Accessibility Many accessibility testing tools are available Tools WebXact (formally known as Bobby) is probably best known http://www.wave.webaim.org/ The WAVE and Cynthia Says are other accessibility checkers NOTES Automated tools can't detect all (many?) accessibility problems Findings from tools can be inconsistent Underlying WAI guidelines are open to interpretation NOTES Automated tools can't detect all (many?) accessibility problems Findings from tools can be inconsistent Underlying WAI guidelines are open to interpretation

6. A centre of expertise in digital information managementwww.ukoln.ac.uk 6 Tools: Usability Of The Tools (1) There can be usability barriers to regular use of such testing tools: They require going to Web page, copying and pasting URL, etc Sometimes only single pages can be tested Simple solution: On UKOLN Web site can append,tools to any URL to run various tools on page Simple to implement – see QA Focus briefing no. 59 http://www.ukoln.ac.uk/qa-focus/ documents/briefings/,rvalidate http://www.ukoln.ac.uk/qa-focus/ documents/briefings/,rvalidate Tools:,validate,rvalidate,checklink,rchecklink,cssvalidate… Tools:,validate,rvalidate,checklink,rchecklink,cssvalidate…

7. A centre of expertise in digital information managementwww.ukoln.ac.uk 7 Tools: Usability Of The Tools (2) 'Bookmarklets' and Firefox extensions can make use of tools much easier and provide additional features Web Developer allows: Features disabled Additional information to be provided Tools to be used Checky allows: Validation Link checking Access to misc tools These tools are very useful and their use by all is strongly encouraged

8. A centre of expertise in digital information managementwww.ukoln.ac.uk 8 Quality Assurance The tools aren't sufficient by themselves. Also need: Documented policies: so we know what we're expected to check for Systematic procedures: for checking that we are implementing our policies Corrections made to workflow processes, and not just fixing individual problems In addition it can be useful to have: Audit trails: to spot trends and identify possible problems in workflow processes (e.g. new tools deployed, new staff involved, …) Sharing experiences, so that we and others can learn

9. A centre of expertise in digital information managementwww.ukoln.ac.uk 9 QA Examples (1) Example of QA policies & procedures for file formats Policy:The Web site will use XHTML 1.0 and CSS 2.0 standards Architecture: The Web site will be based on XHTML templates and use of SSIs Monitoring: New and updated pages validated using,validate and,cssvalidate. Every month,rvalidate will be used & record kept Exceptions: HTML derived automatically (e.g. Save As HTML in PowerPoint) need not comply with standards. The files will be stored in a standard directory to enable such files to be excluded from checks. Policy for QA Focus Web site

10. A centre of expertise in digital information managementwww.ukoln.ac.uk 10 QA Examples (2) Example of QA policies & procedures for links Policy for QA Focus Web site Policy:QA Focus will seek to ensure that links are functional. Monitoring: New and updated pages checked using,checklink and,rchecklink. Every month,rchecklink will be used & record kept and quarterly Xenu will be used. Exceptions: Links in "publications" (e.g. papers which are formally published) which become broken may not be fixed. If there are large numbers of broken links which would be time-consuming to fix we may not fix them. We make no commitment to fix broken links once the QA Focus funding finishes.

11. A centre of expertise in digital information managementwww.ukoln.ac.uk 11 Conclusions To conclude: Tools can help in identifying problems areas However tools may be flawed and inconsistent In order to make tools easier to use they can be accessed: (a) using a URL interface or (b) within the browser Systematic use as part of a QA framework is desirable Note, however, that is essential that you do not rely purely on automated tools – you will need to engage with your users

12. A centre of expertise in digital information managementwww.ukoln.ac.uk 12 Questions Any questions or comments?