1. A centre of expertise in digital information managementwww.ukoln.ac.uk Quality Assurance For Your Web Site Brian Kelly UKOLN University of Bath Bath Email B.Kelly@ukoln.ac.uk URL http://www.ukoln.ac.uk/ UKOLN is supported by: Slides available at <http://www.ukoln.ac.uk/ web-focus/events/workshops/pub-lib-2004/> Slides available at <http://www.ukoln.ac.uk/ web-focus/events/workshops/pub-lib-2004/>

2. A centre of expertise in digital information managementwww.ukoln.ac.uk 2 About Me / QA Focus Brian Kelly: UK Web Focus: adviser on Web standards and best practices Funded by JISC (and MLA 1 Aug 2003) Web developer since 1993 Based at UKOLN QA Focus: Project funded by JISC to support JISC's digital library programmes Has developed a quality assurance methodology and range of support materials Provide by UKOLN and AHDS Project manager is Brian Kelly

3. A centre of expertise in digital information managementwww.ukoln.ac.uk 3 What Can Go Wrong? Problems The problems may be due to: People Technologies Systems The problems may be due to: People Technologies Systems Broken forms Broken links Broken scripts Corrupted display Usability problems Accessibility problems Ambiguous content Out-of-date content Incorrect content Incomprehensible content Device problems Interoperability problems Why Do Things Go Wrong? What Can Go Wrong?

4. A centre of expertise in digital information managementwww.ukoln.ac.uk 4 Finding Errors How do we spot such errors? Automated tools Manual checking User feedback Failure of systems to work correctly Failure of systems to be interoperable … But: How systematic are we in checking? Do users really give us feedback? Do we know when are systems are non- interoperable? What Can Go Wrong?

5. A centre of expertise in digital information managementwww.ukoln.ac.uk 5 Fixing Errors If we spot errors or errors are reported what are our approaches to correcting the errors: Fix them straight away Scope the extent of problems and make plans for fixing problems Do nothing – there are too many errors to fix Do nothing – it's somebody else's responsibility Do nothing – it's a problem with the CMS, the user's browser, … Something else What Can Go Wrong?

6. A centre of expertise in digital information managementwww.ukoln.ac.uk 6 Why Do Things Go Wrong? Things can go wrong for several reasons: Failure to understand the bigger picture: Importance of open standards Limitations of open standards Use of an inappropriate for the deployment of solutions Failure to check compliance with standards Failure to appreciate limitations of testing tools Failure to understand what should be tested What Can Go Wrong?

7. A centre of expertise in digital information managementwww.ukoln.ac.uk 7 QA Focus Approach QA Focus approach to these issues: Advice on reasons for use of open standards Advice on specific open standards Case studies describing approaches taken by projects (including any problems experiences and lessons learnt) Advice on approaches to testing Development of a quality assurance methodology Quality Assurance

8. A centre of expertise in digital information managementwww.ukoln.ac.uk 8 A QA Approach Quality Control: Spotting errors and then fixing them CF production line processes (rejection of bottles, cars, … which aren't up to scratch) Quality Assurance: Having documented procedures Addressing the underlying causes of problems Fixing the workflow processes Addressing human issues (training, …) Introduced in the 1940s Quality Assurance

9. A centre of expertise in digital information managementwww.ukoln.ac.uk 9 We Need Policies Quality Assurance requires documented policies: How do we know if something (non-trivial) is broken if we haven't got a documented policy The policies should be realistic In a Web context we need policies on: HTML compliance CSS Links Accessibility / usability Error reporting … It is recognised that policies may need to be lightweight and not too onerous to develop. Quality Assurance

10. A centre of expertise in digital information managementwww.ukoln.ac.uk 10 We Need Checking Procedures Quality Assurance requires systematic procedures for ensuring compliance with policies: Without this, our policies can be meaningless 'motherhood and apple pie' statements The procedures: Should be systematic Should provide an audit trail Should result in action if deviations from policies found It is recognised that procedures may need to be lightweight and not too onerous to implement. Quality Assurance

11. A centre of expertise in digital information managementwww.ukoln.ac.uk 11 A Template Policy Area: Give the area covered by the policy Standards / Best Practices: State the standards or best practices which will be used Justification: Give reasons for chosen standards / best practices Exceptions: State any permitted deviations Implementation Architecture: If applicable, describe the architecture used to implement the standards Change Control: Describe the responsibilities for the policy, its implementation and for making changes to the policy. Quality Assurance

12. A centre of expertise in digital information managementwww.ukoln.ac.uk 12 A Template Procedure Area: Give a link to the policy. Procedure(s): Describe the procedure(s) used for ensuring compliance with the policy. Limitations: Describe any limitations in the procedures. Audit Trails: Describe any audit trails used to record the findings of the procedures. Correcting Errors : Describe the approaches for correcting errors which may be found. Change Control: Describe the responsibilities for the procedures, its implementation and for making changes to the procedures. Quality Assurance

13. A centre of expertise in digital information managementwww.ukoln.ac.uk 13 QA Areas Areas in which QA Focus has been developing QA policies and procedures and accompanying support materials include: Web/access  Digitisation Metadata  Software Service Deployment  Standards selection Quality Assurance

14. A centre of expertise in digital information managementwww.ukoln.ac.uk 14 QA For Web QA for Web sites will cover areas such as: HTML compliance CSS compliance Functionality in Web browsers Link checking Accessibility checking Usability checking Accuracy of content … Quality Assurance

15. A centre of expertise in digital information managementwww.ukoln.ac.uk 15 QA For Web (HTML/CSS) - Policy Area: Web standards (HTML, CSS). Standards / Best Practices: Web site will comply with XHTML 1.0 and CSS 2.0. Justification: Compliance with HTML &CSS standards with help to maximise access to Web site. Exceptions: Files derived from MS Office apps need not comply with HTML standard. Implementation Architecture: PHP on Apache platform used, which includes HTML fragments. Also makes use of backend MS SQL Server database and MT Blog. Change Control: The project manager is responsible for the policy, its implementation and for making changes to the policy. Quality Assurance

16. A centre of expertise in digital information managementwww.ukoln.ac.uk 16 QA For Web (HTML/CSS) - Procedures Area: Web standards (HTML, CSS). Procedure(s): The,validate tool should be used when pages created/updated. The,rvalidate tool should be used at least quarterly. The W3C Web Log validator should be used monthly. Limitations: The,rvalidate tool only validate up to 100 files. The W3C Web Log validator only validates the 10 most popular pages. Audit Trails: An audit trail will be kept of the output from the monthly W3C Web Log validator output. Correcting Errors : Errors spotted using,validate and,rvalidate should be updated immediately. A record of pages fixed/not fixed should be kept for the W3C Web Log validator output. Quality Assurance

17. A centre of expertise in digital information managementwww.ukoln.ac.uk 17 QA In Other Web Areas CSS: Similar to HTML standards (see briefing doc) Link-checking: Need systematic use of link-checkers. Need to ensure tools covers links other than and e.g. links to CSS & JavaScript files. Need to have policy on fixing broken links. Accessibility: Important to have QA to cover "reasonable measures" clause in DDA. Will need automated and manual checks. Usability: Related to accessibility. Will need automated and manual checks. Quality Assurance

18. A centre of expertise in digital information managementwww.ukoln.ac.uk 18 QA And Metadata Metadata is the glue for interoperable services. It is therefore important that we have QA to ensure that our metadata is Accurate Represented in correct format Interoperable with other services For further information see: An Introduction To Metadata (briefing 41) Metadata Deployment (briefing 42) Quality Assurance For Metadata (briefing 43) See Quality Assurance

19. A centre of expertise in digital information managementwww.ukoln.ac.uk 19 QA And Software Software may be: Used software to create, manage and deliver resources on our Web site Purchased, developed locally or used as open source There is a need to: Ensure software is appropriate for its purpose Ensure we have resources needed to use / develop software Ensure software outputs are compliance with appropriate standards & guidelines See Quality Assurance

20. A centre of expertise in digital information managementwww.ukoln.ac.uk 20 QA And Service Deployment Project-funded work can help to develop content, applications, etc. which will then be deployed in a service environment. There is a need to ensure that project deliverables: Can be deployed easily Are legal and unencumbered with IPR restrictions … See Quality Assurance

21. A centre of expertise in digital information managementwww.ukoln.ac.uk 21 Deviation From Best Practices QA is about “fitness for purpose” – not necessarily the ideal solution. The NOF-digitise Technical Advisory Service defined a reporting process when non-optimal solutions (e.g. proprietary formats like Flash) were to be deployed: Description of preferred open standard/best practice Proposed solution Reason for choice of proposed solution Description of migration strategy A NOF-TAS FAQ gives scenarios such as use of Flash and use of externally-hosted Web services Quality Assurance

22. A centre of expertise in digital information managementwww.ukoln.ac.uk 22 Matrix For Standards Selection The selection of formats to be used is not necessarily easy. Open standards may be immature, costly to deploy or fail to be widely deployed (cf. OSI networks) We have developed a template matrix for selection: Maturity of standard Complexity Availability of tools Resource issues Organisational culture Quality Assurance

23. A centre of expertise in digital information managementwww.ukoln.ac.uk 23 Useful Tools A number of simple tools and techniques for checking compliance have been documented:,tools Append,validate,rvalidate,checklink etc. to any URL on UKOLN Web site Easy to implement – see,tools W3C QA Log Validator Periodic report on 10 most popular pages which are non-compliant Means of prioritising pages to fix (and spotting workflow problems and motivating page authors to address problems) Simple Perl script See Quality Assurance

24. A centre of expertise in digital information managementwww.ukoln.ac.uk 24 QA In "Softer" Areas There may be a temptation to address only the ‘hard‘ areas with use of automated tools It is equally important to address ‘softer’ areas such as accessibility, usability, content, functionality, etc. (cf. DRC Accessibility Report) How can QA be used in these areas: Still a need for policies Testing compliance cannot be done with automated tools See Alice Grant’s report on approaches to evaluation – to be published on MLA Web site shortly Sarah Agarwal's talk on usability testing Quality Assurance

25. A centre of expertise in digital information managementwww.ukoln.ac.uk 25 Embedding QA In Your Library QA Focus resources: Developed for JISC digital library community Looking to extend remit to include MLA sector You can help by providing feedback on: Existing resources QA methodology What’s missing … Please complete feedback forms and return Quality Assurance

26. A centre of expertise in digital information managementwww.ukoln.ac.uk 26 Conclusions To conclude: Web sites now provide mission-critical services Robustness and reliability are therefore crucial We could react to problems A better approach is use of well-established quality assurance principles QA need not be onerous to introduce QA Focus have developed a methodology and accompanying materials which are freely available Conclusions