1. A centre of expertise in digital information managementwww.ukoln.ac.uk QA For Web Sites Brian Kelly UKOLN University of Bath Bath Email B.Kelly@ukoln.ac.uk URL http://www.ukoln.ac.uk/ UKOLN is supported by: Contents What Is Quality Quality And Web Sites QA Case Study Approaches To Testing Benchmarking Conclusions Contents What Is Quality Quality And Web Sites QA Case Study Approaches To Testing Benchmarking Conclusions

2. A centre of expertise in digital information managementwww.ukoln.ac.uk 2 Background To Web & QA Focus Web Focus: A JISC-funded post to advise the UKs HE and FE community Provided by Brian Kelly QA Focus: An advisory service for JISCs 5/99 projects Provides advice and support for projects in order to ensure that projects comply with standards and best practices Provided by UKOLN (Marieke Guy & Brian Kelly) and ILRT This talk is informed by the work of Web and QA Focus

3. A centre of expertise in digital information managementwww.ukoln.ac.uk 3 Quality Control and Quality Assurance Quality Control (QC): Use of regular testing procedures against your definitions of quality and more specifically the refinement of these procedures Relates to outputs Quality Assurance (QA): examines the processes that shape your Web site in the first place and so encompasses Quality Control Relates to inputs Should be involved in development stage of a Web site and throughout its life cycle

4. A centre of expertise in digital information managementwww.ukoln.ac.uk 4 What Can Fail on Your Site? Broken links, broken emails Server load – too many hits on the site Client side performance –down load time Security isnt working Content is out of date Browser incompatibility,HTML doesnt validate Interface – navigation, link colour Graphics missing or too large Scripts dont work - forms, databases Isnt accessible to those with disabilities Browser dependencies …

5. A centre of expertise in digital information managementwww.ukoln.ac.uk 5 What Can You Test? Functional testing Compatibility testing Load/performance testing Stress testing Usability testing Security testing Integration of unit testing Link testing HTML Validation Reliability testing Regression testing …

6. A centre of expertise in digital information managementwww.ukoln.ac.uk 6 What Procedures? Good documentation Requirements Specifications Mission statements or statement of direction Define your audience Testing suite and tools Usability testing Use the right authoring tools etc. Track problems

7. A centre of expertise in digital information managementwww.ukoln.ac.uk 7 Issues Things to be aware of: Testing goals should relate to kind of site you have Testing time is limited use automated tools Automated tools can be inadequate use manual tests Documentation is critical to ensure practices are repeatable Motivating/persuading people to implement a QA culture can be difficult Implement your QA procedures over time to avoid stifling productivity QA planning at the start of development means less time fixing things at the end

8. A centre of expertise in digital information managementwww.ukoln.ac.uk 8 Case Study: IWMW Web Site http://www.ukoln.ac.uk/web-focus/events/workshops/webmaster-2002/ Case Study

9. A centre of expertise in digital information managementwww.ukoln.ac.uk 9 Aims Of Web Site The aims of the Web site were: To provide information about the workshop for participants, speakers, etc. To provide an online booking facility To act as a demonstrator of standards, best practices and innovative solutions Subject to the following constraints: No Content Management System Limited time Limited software development / design expertise Case Study

10. A centre of expertise in digital information managementwww.ukoln.ac.uk 10 Approach The approach taken was to use: The design and structure used for previous two workshops The HTML-Kit authoring tool to edit HTML files The Xenu link checker Innovations this year include: Use of HTML and CSS validation icons Provide an XHTML Web site Make greater use of CSS Provide a dynamically-generated RSS news feed Provide live usage statistics Access to Web site on PDAs using AvantGo Is this approach typical of yours – incremental development of an existing Web site, and introducing new features and more up-to-date standards? Case Study

11. A centre of expertise in digital information managementwww.ukoln.ac.uk 11 Tools HTML-kit is my preferred editor (and recent version is even better, providing batch validation) The XHTML and CSS icons act as live links The SiteMeter icon gives realtime info on usage The XHTML and CSS icons act as live links The SiteMeter icon gives realtime info on usage

12. A centre of expertise in digital information managementwww.ukoln.ac.uk 12 Problems Some problems: When updating pages, links, HTML and spelling were not checking systematically When using the HTML and CSS checker the browser could check an old, cached version At one stage the SiteMeter usage statistics service was not available Case Study It can be a useful (and therapeutic!) process to talk about problems with your Web site.

13. A centre of expertise in digital information managementwww.ukoln.ac.uk 13 Validating In Batch Checking (e.g. HTML and CSS validation, links, accessibility, etc.) initial files is time-consuming and, for large or continually updated Web sites is not a scalable solution Validation in batch: Familiar for link-checking Limited set of tools available for HTML validation (e.g. HTML Validator Pro> Batch HTML validation validates files generated from PowerPoint, over which I have no control Bobby batch accessibility check is now a licensed application – so used old copy Case Study Can now use HTML-Kit for validating the Web site.

14. A centre of expertise in digital information managementwww.ukoln.ac.uk 14 Netscape (sigh) User feedback revealed that: £ does not display in Netscape :-( The Suggestions text box and the Submit button (!) were not displayed in versions of Netscape, due to failure to support XHTML correctly Suggestions Submit Case Study

15. A centre of expertise in digital information managementwww.ukoln.ac.uk 15 RSS News Feed The RSS News Feed: Provides a mechanism for syndication of content News feed can be viewed on other Web sites or in other applications The News Feed was created: By transforming (lightly structured) HTML from the news page using a remote service A static version was also created in case of problems with the live transformation Problems: The remote service was not always available The static version was not always in sync with the live version Case Study Subsequently the dynamic transformation service became unavailable

16. A centre of expertise in digital information managementwww.ukoln.ac.uk 16 Online Bookings The online booking forms: Were used for booking for the workshop and choosing the parallel sessions The backend processing: Is not yet fully automated to store the information in a database Confirmation page did not echo the users data When users followed link to check session, input data may have been lost Crashed on surnames with spaces and accents No validation was performed Despite messages on acknowledgements page, some users still expected a separate email confirmation Case Study

17. A centre of expertise in digital information managementwww.ukoln.ac.uk 17 What Should Be Done Some simple things can be done for next time: Use of SSIs or a CMS (or equivalent ) to manage resource fragments Improve the online booking system to provide: Validation (which will save time in administration) Display of user input Separate confirmation of entry into database But: I recognise that minor typos, etc. will happen and, as long as they are not critical, can be tolerated The goal is quality – not perfection Case Study

18. A centre of expertise in digital information managementwww.ukoln.ac.uk 18 What About The Processes? What QA processes were used? Automated tools Peer review from members of the organising committee Alternating between Opera, IE and Mozilla when working on Web site Viewing the Web site on my Palm to: Allow me to work on the bus, train, etc. See how the Web site works on a low-spec device See < http://www.ukoln.ac.uk/web-focus/ events/workshops/webmaster-2002/qa/ > See < http://www.ukoln.ac.uk/web-focus/ events/workshops/webmaster-2002/qa/ > Case Study

19. A centre of expertise in digital information managementwww.ukoln.ac.uk 19 What Else Should I Do? Additional processes I should employ: Reviewing by users Peer-reviewing (others in similar situation) Analysis of logs (esp. failure logs) Usability testing Multiple browser testing Test data with unusual entries for online forms Post-workshop review of Web site Systematic documentation of successes and failures Make case for extra resources, software, etc. to implement improvements Documenting the architecture of the Web site Documenting the limitations of the Web site Preparing these slides! … Case Study

20. A centre of expertise in digital information managementwww.ukoln.ac.uk 20 Automated Testing Automated Testing: Should be scalable Only suitable approach for large Web sites Can push (email) information about problems Many free automated testing tools available but: Will not spot all usability / accessibility problems Can provide too much information Comprehensive and configurable testing tools can be expensive

21. A centre of expertise in digital information managementwww.ukoln.ac.uk 21 Manual Testing Manual Testing: Can spot usability / accessibility problems Can make use of ones community / user base / peers Can use of key pages which will identify problems which occur elsewhere Particularly useful for testing new Web sites 5 users can spot most of your problems but: Not suitable for testing large numbers of pages Quality of feedback may be variable Often not useful for testing established Web sites

22. A centre of expertise in digital information managementwww.ukoln.ac.uk 22 Approaches To Automated Testing Automated testing can make use of: Desktop tools Web-based tools UKOLN has made use of Web-based testing tools: Monitor accessibility, HTML, CSS, compliance, page size, links quality, nos. of links, etc. Benchmarking approach provides comparison with ones peers See, for example,

23. A centre of expertise in digital information managementwww.ukoln.ac.uk 23 Interfaces to Testing Tools (1) Bookmarklets Embed a testing service into your browser Available for IE, Netscape, Opera, …browsers Article at See or Google search for bookmarklets, accessibility bookmarklets, etc.

24. A centre of expertise in digital information managementwww.ukoln.ac.uk 24 Bookmarklets Example Bookmarklets can be accessed: From the browsers normal Bookmarks / Favourites Menu From a side bar By right-clicking the mouse button

25. A centre of expertise in digital information managementwww.ukoln.ac.uk 25 Interfaces to Testing Tools (2) URL Interface UKOLNs Web site has an interface to various tools provided at the server Easier to maintain See for overview

26. A centre of expertise in digital information managementwww.ukoln.ac.uk 26 Benchmarking (1) Benchmarking: We have discussed various tools for checking ones own site But how do we compare with our peers? Can we learn from others best practices? Can we avoid making mistakes that others have made? Can the methodology to be described be used across your community – e.g. On a regional basis On candidates for Best Public Library Web Site award By government auditors for checking e-GIF compliance Can the methodology to be described be used across your community – e.g. On a regional basis On candidates for Best Public Library Web Site award By government auditors for checking e-GIF compliance

27. A centre of expertise in digital information managementwww.ukoln.ac.uk 27 Benchmarking (2)

28. A centre of expertise in digital information managementwww.ukoln.ac.uk 28 QA Focus Web Site Benchmarking QA Focus: Carried out survey of 43 project Web sites Findings freely available Methodology can be applied to other sectors Where best practices found, projects asked to provide a case study Where problems found, appropriate advice provided

29. A centre of expertise in digital information managementwww.ukoln.ac.uk 29 HTML Compliance W3Cs HTML Validation service is used to record the HTML compliance of project home pages http://www.ukoln.ac.uk/qa-focus/surveys/

30. A centre of expertise in digital information managementwww.ukoln.ac.uk 30 Accessibility Bobby is used to record compliance with W3C WAI guidelines: 24 comply with A 5 comply with AA 21 are not compliant Note that only compliance with guidelines which can be monitored by automated tools is recorded

31. A centre of expertise in digital information managementwww.ukoln.ac.uk 31 Repurposing Resources Are pages from project Web sites available in the Internet Archive? Can the project Web site be accessed on a PDA?

32. A centre of expertise in digital information managementwww.ukoln.ac.uk 32 404 Pages 404 error pages can provide an important navigational feature for Web sites Tailored 404 Page1433% Default 404 Page2867% * Rating321% ** Rating321% *** Rating857% **** Rating00% * Basic **Simple branding ***Additional functionality ****Full functional

33. A centre of expertise in digital information managementwww.ukoln.ac.uk 33 Critique Limitations of this approach include: Project Web sites may be: The project deliverable About the project For intra-project communications The project deliverable may be, say, middleware Limitations of the analysis tools Limited number of pages surveyed …

34. A centre of expertise in digital information managementwww.ukoln.ac.uk 34 Supporting Best Practices QA Focus is supporting JISC 5/99 projects by providing advice on compliance with standards and best practices, based on the survey findings http://www.ukoln.ac.uk/ qa-focus/documents/ http://www.ukoln.ac.uk/ qa-focus/documents/

35. A centre of expertise in digital information managementwww.ukoln.ac.uk 35 Case Studies QA Focus is commissioning case studies: Document approaches projects have taken in particular areas Describe the problem area, the approach taken, problems experienced and lessons learnt

36. A centre of expertise in digital information managementwww.ukoln.ac.uk 36 Conclusions To conclude: Quality assurance is important It will grow in importance: e.g. compliance with e- GIF guidelines, accessibility legislation, etc. Automated tools can help Manual testing is needed to complement automated approaches Benchmarking can provide information on approaches across communities