1. QA Focus – Supporting JISC's 5/99 Programme QA For Digital Library Projects Brian Kelly UKOLN B.Kelly@ukoln.ac.uk http://www.ukoln.ac.uk/qa-focus/ This talk will describe the work of the JISC- funded QA Focus team and outlines approaches to QA which can be deployed in Digital Library projects

2. QA Focus – Supporting JISC's 5/99 Programme Contents Background: The need for QA QA Focus Work activities: Web Accessibility Providing Advice Future Work Metadata Software Development Service Deployment Toolkit Synergies With Other Groups Conclusions

3. QA Focus – Supporting JISC's 5/99 Programme Aims Of Todays Talk To describe the background to the QA Focus project To summarise QA Focus activities to date To describe future activities To explore potential for links with CDLR EIB Introduction

4. QA Focus – Supporting JISC's 5/99 Programme What is Quality? Quality is the ability of your product to be able to satisfy your users What is Quality Assurance? Quality assurance is the process that demonstrates your product is able to satisfy your users An Introduction to QA Introduction

5. QA Focus – Supporting JISC's 5/99 Programme An Introduction to QA What does Quality Assurance give? Quality means your project is useful - without quality you may have little to offer Quality can help to future-proof projects But quality assurance needs documented standards and best practices to be meaningful Quality & Best Practice can only be considered in terms of being Fit for Purpose Introduction

6. QA Focus – Supporting JISC's 5/99 Programme An Introduction to QA Why Is QA Needed Now? In the past JISC had standards-based philosophy but: Approach of "let a 1,000 flowers bloom" Diversification as no killer application (it could have been Gopher!) Things are different now: The Web is the killer application There is now a need for interoperability (now self- contained independent Web sites) There is a need for publicly-funded services to be seen to provide value for money Introduction

7. QA Focus – Supporting JISC's 5/99 Programme Approach Taken Two possible approaches to ensuring compliance with standards and best practices: Enforce Inspect all projects work Strict auditing, with penalties for no-compliance Encourage Training for project staff Developmental, explaining reasons for compliance, documenting examples of best practices and providing advice on implementation and monitoring Introduction The latter approach is preferable, especially in a HE context

8. QA Focus – Supporting JISC's 5/99 Programme QA Focus QA Focus: Funded by JISC Currently provided by UKOLN and AHDS Currently supports JISC's 5/99 programme Staff: Brian Kelly, Project manager Marieke Guy, QA Focus officer, UKOLN Hamish James, QA Focus officer, AHDS

9. QA Focus – Supporting JISC's 5/99 Programme QA for Digitisation Do it once…..do it right: Project is fundamentally dependent upon the quality of original product Quality is the pre-requisite to preservation Quality expectations will only grow Delivery problems can be fixed, but capture problems normally cant QA For Digitisation

10. QA Focus – Supporting JISC's 5/99 Programme QA Philosophy A multi-level approach may be taken to QA: Strategic QA Carried out before digitisation starts Research and establishing best practice & standards QA For Digitisation

11. QA Focus – Supporting JISC's 5/99 Programme QA Philosophy A multi-level approach may be taken to QA: Strategic QA Workflow QA Formative assessment, before & during development Establishing & documenting workflow & processes QA For Digitisation

12. QA Focus – Supporting JISC's 5/99 Programme QA Philosophy A multi-level approach may be taken to QA: Strategic QA Workflow QA Sign-off QA Quality Control : Summative assurance at end of each process, providing an audit history for all QA work undertaken QA For Digitisation

13. QA Focus – Supporting JISC's 5/99 Programme QA Philosophy A multi-level approach may be taken to QA: Strategic QA Workflow QA Sign-off QA On-going QA Summative assurance as part of long term QA to establish a system to report, check & fix any faults found in future QA For Digitisation

14. QA Focus – Supporting JISC's 5/99 Programme QA Philosophy QA Focus promotes a multi-level approach to digitisation: Strategic QA Workflow QA Sign-off QA On-going QA High Quality Product QA For Digitisation

15. QA Focus – Supporting JISC's 5/99 Programme QA For Web Sites The issues: The Web is the main delivery mechanism for projects and services An increasing awareness of the importance of: Accessibility Use of new devices (PDAs, WAP, e-books, …) Repurposing of Web content (e.g. archiving) Technologies such as XSLT will support repurposing of valid XML resources But: Invalid HTML is the norm Many authoring tools produce poor HTML Authors arent aware of the problems QA For Web

16. QA Focus – Supporting JISC's 5/99 Programme Guidelines We often say: Open standards are important HTML, XML, XHTML, CSS, … are important but fail to explain why and how JISCs QA Focus is addressing such concerns by: Documenting example of best practices in which projects can share their implementation successes (and difficulties they experienced) Provide brief advice in specific aspects of the standards and best practices Surveying its communities to highlight best practices and areas in which improvements can be made Demonstrating use of testing tools and procedures QA For Web

17. QA Focus – Supporting JISC's 5/99 Programme Standards & Best Practices Standards For Web: Use compliant HTML / XHTML Use CSS Support WAI accessibility guidelines Best Practices For Web: Ensure Web resources are suitable for reuse and repurposing Where proprietary formats need to be used, flag them and use in most open way QA For Web

18. QA Focus – Supporting JISC's 5/99 Programme Surveying The Community Surveys of project Web sites have been carried out in order to: Obtain a profile for the community Identify examples of best practices Identify areas in which further advice is needed Surveys included: HTML & CSS compliance Accessibility 404 error pages HTTP headers Repurposing resources QA For Web

19. QA Focus – Supporting JISC's 5/99 Programme Survey Philosophy The surveys made use of freely-available Web-based tools: Methodology is open No software needs to be installed locally (apart from Web browser) Findings can be reproduced Latest results can be obtained by clicking on link to testing service The surveys typically examined project entry points and not entire Web site as: This page has the highest profile The aim is to validate a methodology which can be deployed by projects themselves, not to test every page on behalf of the projects QA For Web

20. QA Focus – Supporting JISC's 5/99 Programme Survey Findings Initial set of findings available from QA For Web

21. QA Focus – Supporting JISC's 5/99 Programme Providing Motivation We have found evidence of failure to comply with HTML standards There is a need to explain why compliance is important (and avoid the its OK in my browser argument) and to provide motivation for projects to update their tools, authoring procedures, etc. A further set of surveys look at repurposing of the project Web sites: Availability of Web sites in the Internet Archive Ease of making Web sites available on a PDA Transformation of embedded metadata QA For Web

22. QA Focus – Supporting JISC's 5/99 Programme Repurposing Resources A small number of Web sites were not in the Internet Archive due to the robots.txt file. We will need to provide advice in this area. We examined the Web sites to see if they were available in the Internet Archive and could be transformed into a format for viewing on a PDA A small number of Web sites could not be transformed. Analysis of HTTP headers indicated that this was due to incorrect HTTP headers. We will need to provide advice in this area. QA For Web

23. QA Focus – Supporting JISC's 5/99 Programme Transforming Resources Project entry points were processed by several online transformation services in order to validate and visualise embedded Dublin Core metadata HTML resource Original page, containing embedded DC metadata Tidy (online) Virtual XHTML resource XSLT extraction of DC DC in RDF format Visualisation & validation of DC RDF Validator QA For Web

24. QA Focus – Supporting JISC's 5/99 Programme Providing Advice We have: Survey project Web sites and identified areas of lack of compliance with standards and best practices Demonstrated examples of the potential importance of compliance for repurposing resources In addition we need to provide: Brief focussed advice on the standards Information on how to monitor compliance Case studies on solutions deployed by projects themselves Guidance on dealing with implementation difficulties and what to do when strict compliance is difficult to achieve Advice

25. QA Focus – Supporting JISC's 5/99 Programme Documentation: Advice Advisory briefing documents are being produced These are: Brief, focussed documents Informed by findings of the surveys Advisory briefing documents are being produced These are: Brief, focussed documents Informed by findings of the surveys Advice

26. QA Focus – Supporting JISC's 5/99 Programme Documentation: Case Studies Case Studies are being commissioned These are: Written by projects themselves Describe the solution adopted to a particular problem Include details of lessons learnt – not just a press release! Case Studies are being commissioned These are: Written by projects themselves Describe the solution adopted to a particular problem Include details of lessons learnt – not just a press release! Advice

27. QA Focus – Supporting JISC's 5/99 Programme Putting it Together Based on surveys we find Project Web sites which: Have invalid HTML and CSS Don't have helpful 404 pages … So we: Explain why projects should follow best practices (e.g. repurposing XHTML pages) Provide case studies on approaches by projects Describe tools and architectures for projects to implement best practices Describe tools and architectures for projects to check compliance with best practices

28. QA Focus – Supporting JISC's 5/99 Programme Describing Tools and Approaches As an example of approach which provides easier access to testing tools, see the,tools approach deployed on UKOLN Web site. This approach: Applies to all resources on Web site Covers HTML and CSS validation and various other tests Some recursive apps (,rvalidate ) Easily implemented with single line redirect As an example of approach which provides easier access to testing tools, see the,tools approach deployed on UKOLN Web site. This approach: Applies to all resources on Web site Covers HTML and CSS validation and various other tests Some recursive apps (,rvalidate ) Easily implemented with single line redirect

29. QA Focus – Supporting JISC's 5/99 Programme Limitations There are a number of limitations to the work we have carried out so far: Project Web sites have different purposes (information about the project; communications with project partners; project deliverables themselves; etc.) Projects have different levels of funding, resources, expertise, etc. Projects are at different stages of development (and some have finished) The surveys are intended to demonstrate a methodology which projects can use for themselves

30. QA Focus – Supporting JISC's 5/99 Programme Next Steps Extended Coverage We will be moving on from Web and digitisation to include other areas including: Metadata Multimedia Software development Deployment into service … Moving On From Automated Testing The initial work made use of automated testing tools: Can be used remotely Objective Applicable across all projects We have started work on QA procedures in areas which are not suitable for automated checking Next Steps

31. QA Focus – Supporting JISC's 5/99 Programme Self Assessment Toolkit Further Deliverables We will be developing a self-assessment toolkit for projects to use, by individual projects or across project clusters The toolkit will consist of: Examples of QA procedures Documented examples of use of testing tools Self-assessment questionnaires Advice on standards and best practices Case studies FAQs … Next Steps

32. QA Focus – Supporting JISC's 5/99 Programme Next Steps – Metadata Thoughts on QA for metadata: SyntaxCheck syntax on embedded DC Cataloguing RulesEnsure projects have appropriate cataloguing rules InteroperabilityEnsure metadata can interoperate with third parties Fitness for purposeEnsure metadata is appropriate for its purpose Change controlEnsure architecture for managing metadata can cope with change control (cf Exploit Case Study) Next Steps

33. QA Focus – Supporting JISC's 5/99 Programme Next Steps – Software Thoughts on QA for software: SpecificationProcess for developing systems spec, agreeing spec with stakeholders, etc. MethodologyDocumentation of systems development methodology (cf UML case study) TestingDocumentation of testing methodology DeploymentAwareness of deployment challenges Next Steps

34. QA Focus – Supporting JISC's 5/99 Programme Next Steps – Service Deployment Thoughts on QA for service deployment: Development Architecture Early public documentation of development architecture Service Architecture Awareness of likely service architecture and liaison with service SecurityAwareness of security concerns LicensingLicencing issues Scenarios Software developed using cool open software environment – but JISC Service has no knowledge of environment. Resources digitised, but copyright clearance not documented. Service refuses to take delivery due to liability concerns. Scenarios Software developed using cool open software environment – but JISC Service has no knowledge of environment. Resources digitised, but copyright clearance not documented. Service refuses to take delivery due to liability concerns. Next Steps

35. QA Focus – Supporting JISC's 5/99 Programme QA Focus And CDLR What relevance has QA Focus for CDLR? You can write case studies and advisory documents: Seen to be working with JISC We can promote your approaches (and therefore your work) This will help instigate best practices within your group

36. QA Focus – Supporting JISC's 5/99 Programme QA Focus And CDLR What relevance has QA Focus for CDLR? You can deploy QA Focus practices for your projects, etc: You should benefit You can provide feedback to QA Focus You can develop QA procedures in innovative areas (e.g. OAI, thesaurii, ePrints, e-Books, etc.) which QA Focus can use: You gain kudos, publications, etc. We can help embed your practices across the community

37. QA Focus – Supporting JISC's 5/99 Programme Applying QA – Standards Policy:Open standards used where possible. Proprietary formats with published spec and cross-platforms viewers and authoring tools may be acceptable Monitoring:Project manager responsible Documentation: Document on standards (cf QA Focus doc) Exceptions:If open standards not available, immature or costly to implement proprietary standards may be used. The project manager should document such decisions: why proprietary format was used, migration strategies to open formats in future, architecture for migration and indication of costs QA Examples

38. QA Focus – Supporting JISC's 5/99 Programme Applying QA – Web Standards Policy:The Web site will use XHTML 1.0 and CSS 2.0 standards Architecture:The Web site will be based on XHTML templates and SSI Monitoring:New and updated pages validated using,validate and,cssvalidate. Every month,rvalidate will be used Exceptions:HTML derived automatically (e.g. Save As HTML in PowerPoint) need not comply with standards. The files will be stored in a standard directory to enable such files to be excluded from checks QA Examples

39. QA Focus – Supporting JISC's 5/99 Programme Applying QA – Proprietary Formats Policy:Proprietary formats may be deployed on Web site in certain documented circumstances including MS Office files, PDF and Flash MS Office files should also be stored in most open HTML format. PDF files should include link to online PDF converter Flash can be used for specific applications in which it is needed. Monitoring:Project manager QA Examples

40. QA Focus – Supporting JISC's 5/99 Programme Applying QA – Accessibility Policy:The Web site will strive to attain WAI A guidelines Consistent accessibility shortcuts will be used An accessibility policy will be published Architecture:The Web site will be based on XHTML templates which comply with WAI A Monitoring:New and updated pages will be validated using,bobby. Every month a batch checker will be used. Monthly audit reports will be published (to enable any trends to be spotted) Exceptions:A list of permitted exceptions will be provided QA Examples

41. QA Focus – Supporting JISC's 5/99 Programme Applying QA – Usability (1) Policy:A structured usability test will be applied using at least 5 users for new Web sites Tailored 404 error pages will be provided Use of to support navigation across Web site Use of to support navigation in structured areas of Web site Architecture:The elements should be implemented by automated scripts Monitoring:Need to check the next, previous goes to correct page. Exceptions:… QA Examples

42. QA Focus – Supporting JISC's 5/99 Programme Applying QA – Usability (2) Policy:Broken links to be kept to a minimum Monitoring:The,checklink command applied to new and updated resources and,checklink run monthly The Xenu link checker will be run monthly and a report published, in order for any trends to be monitored Exceptions:Don't fix broken links in published documents or derived files. Need to formulate policy on large numbers of broken links (cf redesign of JISC Web site) QA Examples

43. QA Focus – Supporting JISC's 5/99 Programme Questions Any questions?