1. A centre of expertise in digital information managementwww.ukoln.ac.uk Institutional Web Management Workshop 2004: Transforming The Organisation Brian Kelly UKOLN University of Bath Bath B.Kelly@ukoln.ac.uk UKOLN is supported by: QA For Web Sites – What Goes Wrong And How Can We Prevent It? QA Focus Web Site:

2. A centre of expertise in digital information managementwww.ukoln.ac.uk 2 Draft Timetable About This Session What Can Go Wrong (group exercise 1) About QA Applying QA (group exercise 2) QA Focus Methodology Further QA (group exercise 3) QA Focus approaches Embedding QA (individual/group exercise 4) Conclusions 16:00 16:10 16:20 16:30 16:45 17:00 17:15 17:25 17:30

3. A centre of expertise in digital information managementwww.ukoln.ac.uk 3 About The Session Aims of the session: To explain how quality assurance (QA) procedures can help to minimise problems To describe how QA can be used in the context of Web sites To outline QA resources which have been developed by the JISC-funded QA Focus project To explore the potential for embedding QA into your Web development work Talk 1

4. A centre of expertise in digital information managementwww.ukoln.ac.uk 4 About Us Brian Kelly: UK Web Focus Project Manager for QA Focus Amanda Closier: Working on QA Focus since Jan 2004 Involved in other JISC project work This session will be based on our QA Focus experiences. We will primarily be addressing technical issues, although non-technical issues may be raised. Talk 1

5. A centre of expertise in digital information managementwww.ukoln.ac.uk 5 Exercise 1 In small groups: Introduce yourselves Outline key areas in which things can go wrong on your Web site Group these into:  Mission-critical problems  Significant problems  Minor problems Choose a reporter to give a brief report back Spend about 10 minutes on this exercise.

6. A centre of expertise in digital information managementwww.ukoln.ac.uk 6 What Can Go Wrong? Broken links Pages not displayed Pages displayed incorrectly Forms don't work Out-of-date content Incorrect or misleading content Poor performance Inaccessible Web site Usability problems Security problems Possible Problems People issues Workflow issues Backend systems don't work Wrong functionality Talk 2

7. A centre of expertise in digital information managementwww.ukoln.ac.uk 7 What Are We Seeking To Achieve? The QA Focus work was based on the notion that projects would be seeking to achieve the following: Functionality of the project deliverables: the deliverables will do what they are supposed to do Wide accessibility for the project deliverables: the deliverables will be accessible in a wide range of environments and to people with a wide range of disabilities Interoperability of the project deliverables: the deliverables will be interoperable with other services, including services not envisaged at the start and services not yet available Ease of deployment of the project deliverables Note that aims of accessibility and interoperability are key QA aims of the Web itself Talk 2

8. A centre of expertise in digital information managementwww.ukoln.ac.uk 8 Why Do Things Go Wrong? Things can go wrong for several reasons: Lack of clarity of the purpose of the service Use of inappropriate standards and technical architecture (possibly due to lack of knowledge) Too high expectations Failure to monitor developments Thinking things are going fine when they're not Lack of formal checking procedures Limitations in checking procedures … Talk 2

9. A centre of expertise in digital information managementwww.ukoln.ac.uk 9 About Quality Assurance Quality Control: Spot an error and you fix it Fire-fighting Failure to learn Quality Assurance: Documented policies (you know what you are seeking to achieve) Systematic compliance checking procedures Audit trails Learning from experiences Part 2: About QA Talk 2

10. A centre of expertise in digital information managementwww.ukoln.ac.uk 10 Exercise 2: Simple Web QA In your groups: Summarise simple QA policies for the technical aspects of your Web site Outline appropriate compliance testing procedures Exercise 2 Spend about 10 minutes on this exercise.

11. A centre of expertise in digital information managementwww.ukoln.ac.uk 11 Simple Web QA QA Focus has developed simple QA policies and procedures covering HTML & CSS formats and links Policy: Web Standards Standard: XHTML 1.0 and CSS 2.0 Architecture: Use of SSIs and text editor Exceptions: Automatically-derived files Checking: Use,validate after update Audit Trail: Use,rvalidate monthly and document findings Policy example Please note that this is a template – you can add additional headings Talk 3

12. A centre of expertise in digital information managementwww.ukoln.ac.uk 12 Make Testing Simple You should aim to ensure that checking compliance with your policies is simple to achieve We recommend the,tools approach for: HTML & CSS compliance Link checking Recursive checking … This approach is: Simple to implement (redirect in Apache conf. file) Requires no additional software Can be used by anyone Talk 3

13. A centre of expertise in digital information managementwww.ukoln.ac.uk 13 Fitness For Purpose Aim for fitness for purpose and not perfection: It would be nice to have compliant HTML version of PowerPoint slides But MS's HTML conversion creates dodgy HTML Rather than (a) not having resources available or (b) wasting time fixing errors we can define our policy accordingly Note that: We do ensure that the HTML version is acceptable to all browsers If a better conversion tool becomes available we can tighten up our policy Talk 3

14. A centre of expertise in digital information managementwww.ukoln.ac.uk 14 Limitations Of Testing Sometimes testing tools can be flawed: Bobby will not tell you if your site is acceptable (you need manual testing to do that) Link validators: Sometimes only check for and tags – and fail to spot problems in tags to external style sheets, JavaScript files, etc. Proprietary processing in MS XHTML gives misleading broken link for MS PowerPoint files Might not be able to access personalised pages (user agent negotiation, environment factors, …) Server errors, proxies, etc may give mistaken broken links Talk 3

15. A centre of expertise in digital information managementwww.ukoln.ac.uk 15 The QA Approach The QA approach: Simple and understandable Achievable! Recognises and addresses real world difficulties Not a specification for a preferred solution Measurable: Compliance measures Easy to implement Supported by training Limitations of compliance checking procedures documented Talk 3

16. A centre of expertise in digital information managementwww.ukoln.ac.uk 16 Exercise 3: Further Web QA In your groups choose 1 of 3 exercises: QA for accessibility QA for Web site metadata QA for software (selection of, use of and development of software) In addition if you have time give some thought to QA in 'softer' areas such as: QA for Web development processes QA for usability Summarise relevant QA policies and procedures Exercise 2 Spend about 20 minutes on this exercise.

17. A centre of expertise in digital information managementwww.ukoln.ac.uk 17 Report Back Please give us your approaches

18. A centre of expertise in digital information managementwww.ukoln.ac.uk 18 Exercise 3A: QA For Accessibility Web Accessibility Policy A simple, understandable approach would be to say: The University Web site will comply with WAI WCAG AA guidelines But: Is this realistic? How do you measure compliance? Will it drive out other important aspects – e.g. usability, learning objectives, …? Talk 4

19. A centre of expertise in digital information managementwww.ukoln.ac.uk 19 Exercise 3B: QA For Web Site Metadata (1) QA Focus's policies: Purpose of metadata: To pull in appropriate style sheets Resource discovery – potential for fielded searches (author, document type, abstract, etc.) Architecture (management): Metadata defined in PHP fragments ($author="Brian Kelly") and pulled in and transformed to HTML/XHTML Architecture (deployment): Metadata will be deployed as tags embedded in XHTML. Note although this architecture allows for metadata transformation & reuse, it is poor for metadata management & control of content. However the QA must reflect the real system which has been implemented and not a desired alternative. Note although this architecture allows for metadata transformation & reuse, it is poor for metadata management & control of content. However the QA must reflect the real system which has been implemented and not a desired alternative. Talk 4

20. A centre of expertise in digital information managementwww.ukoln.ac.uk 20 Exercise 3B: QA For Web Site Metadata (2) Policy - Purpose: Metadata will be used to (a) define structure of Web site (Documents, Presentations, …) for visual identity and allow potential for fielded searches and (b) to support resource discovery Policy – Content: Structural metadata will be chosen from set of keywords Resource discovery metadata will be based on a brief abstract and a small number of appropriate keywords. A similar style will be used across all resource Talk 4

21. A centre of expertise in digital information managementwww.ukoln.ac.uk 21 Exercise 3B: QA For Web Site Metadata (3) Checking Procedures: Structural metadata will be checking visually to ensure that the correct style sheet is used Resource discovery metadata will be reused in the printed QA Focus handbook. Visual checking of the printed resources will be used to ensure the accuracy and appropriateness of the metadata HTML validation procedures will ensure the tags comply with HTML Learning Experiences: Metadata management to avoid data duplication Tools to spot incorrect structural metadata Searching for common content errors (e.g. null fields, template data, … Talk 4

22. A centre of expertise in digital information managementwww.ukoln.ac.uk 22 Holistic Accessibility UKOLN & TechDis are seeking to develop a holistic approach to accessibility Need to address: Usability IT accessibility Learning objectives Learner's needs (as opposed to universal solutions) Resource issues…. This approach will be explained in talk on Thursday morning Talk 4

23. A centre of expertise in digital information managementwww.ukoln.ac.uk 23 Exercise 3C: QA For Software (1) Choosing Software: Open sources vs licensed issues Possible dangers of sustainability if non- mainstream See Top Tips For Choosing Open Source Software briefing document Creation Tools: Need to ensure that output (e.g. metadata creation tools) complies with appropriate standards Need for systematic checking of output Talk 4

24. A centre of expertise in digital information managementwww.ukoln.ac.uk 24 Exercise 3C: QA For Software (2) Workflow: Need to ensure that workflow does not corrupt data Need to validate output against range of inputs (e.g. MS Windows characters (©, —, £) input to newsfeed authoring systems Web Forms: Need to ensure that Web forms work – esp. if used for critical systems such as online bookings Need for systematic checking – e.g. names with unusual characters; non-standards browsers; JavaScript disabled; … Talk 4

25. A centre of expertise in digital information managementwww.ukoln.ac.uk 25 Exercise 3D: QA For Web Development This are has not been explored by QA Focus Appropriate solutions based on QA model will include: Clear specification: if you don't know what you're supposed to be building you'll have problems Agreement with stakeholders: Will your users want what you intend to build? Structured usability testing: Systematic procedures for testing the usability of your Web site Agreed acceptance procedures: Are users happy with what you've provided; is the work finished? There are clear parallels with QA: the need for documentation The need for procedures to ensure that you've successful achieved your aims There are clear parallels with QA: the need for documentation The need for procedures to ensure that you've successful achieved your aims Talk 4

26. A centre of expertise in digital information managementwww.ukoln.ac.uk 26 Embedding QA We have explained the QA Focus methodology and sought to apply it in a number of areas We now welcome feedback on how your may envisage deploying this approach (if at all) Please complete forms: The QA Methodology Gap analysis - what else could we do Talk 5

27. A centre of expertise in digital information managementwww.ukoln.ac.uk 27 QA Focus Resources The following QA Focus resources area available: Documents addressing digitisation, Web/access, metadata, software, service deployment, … Over 70 briefing documents Over 30 case studies QA toolkits, which help you to address the key QA areas Eight QA Handbooks which package the above The resources are freely available at stable URLs on the Web. We are seeking to provide a Creative Commons licence for the resources to allow you to use and tailor the resources for non- commercial use, provided due acknowledgements given The resources are freely available at stable URLs on the Web. We are seeking to provide a Creative Commons licence for the resources to allow you to use and tailor the resources for non- commercial use, provided due acknowledgements given Talk 5

28. A centre of expertise in digital information managementwww.ukoln.ac.uk 28 Questions Any questions?