1 QA Focus – Supporting JISC's Digital Library Programmes QA For Metadata: The QA Focus Methodology Brian Kelly, UKOLN Supported by Amanda Closier, UKOLN and Gareth Knight, AHDS This session addresses the importance of quality assurance for metadata and describes the QA methodology developed by JISC- funded QA Focus project.

2 QA Focus – Supporting JISC's Digital Library Programmes Background QA Focus has sought to: Help ensure the functionality, wide accessibility and interoperability of JISC-funded projects Provide advice and support materials to projects Recognise the limited resources available to projects Develop a light-weight methodology which can help projects achieve interoperability and compliance with appropriate standards and best practices Develop a self-assessment methodology which recognises the difficulties of external checking

3 QA Focus – Supporting JISC's Digital Library Programmes Why Do Things Go Wrong? Why do project deliverables fail to be: Functional  Widely accessible Interoperable  Easily deployed into service Reasons include: Failure to implement standards & best practices (perhaps due to a lack of understanding) Inappropriate standard / best practice used Use of an inappropriate architecture (tools, workflow, …) Failure to check that standards & best practices are being used correctly Failure to understand limitations of checking tools Having over-optimistic intentions

4 QA Focus – Supporting JISC's Digital Library Programmes QA Focus Methodology The QA Focus methodology: Based on mainstream QA principles Projects should develop written policies on standards, best practices, etc. Projects should deploy systematic procedures for ensuring the policies are implemented Projects should share experiences (good and bad) Note that: The policies should be achievable The procedures may lead to changes in systems, workflow, etc. if non-compliance spotted Audit trails should be kept to help identify trends, provide documented evidence of best practices, etc.

5 QA Focus – Supporting JISC's Digital Library Programmes Policies How do you know what you should do if you don't have documented polices? Policy: Web Standards Standard: XHTML 1.0 and CSS 2.0 Architecture: Use of SSIs and text editor Exceptions: Automatically-derived files Checking: Use,validate after update Audit Trail: Use,rvalidate monthly and document findings Policy example Please note that this is a template – you can add additional headings

6 QA Focus – Supporting JISC's Digital Library Programmes Applying QA – Accessibility (1) Policy:The Web site will strive to attain WAI A guidelines. Consistent accessibility shortcuts will be used. An accessibility policy will be published. Architecture:The Web site will be based on XHTML templates which comply with WAI A. Monitoring:New and updated pages will be validated using,bobby. A monthly batch checker will be used and audit reports published (to enable any trends to be spotted). Exceptions:A list of permitted exceptions will be provided. QA Focus Methodology Note that University of Edinburgh Library have a useful Web accessibility policy

7 QA Focus – Supporting JISC's Digital Library Programmes Applying QA – Accessibility (2) Policy:The organisation has no accessibility policy Authors are free to implement their own accessibility shortcuts (if at all) Architecture:No centralised policy covering authoring tools or architecture will be provided Monitoring:No monitoring will be carried out If you don't have a written policy, the unwritten policy may well be frightening!

8 QA Focus – Supporting JISC's Digital Library Programmes Who Checks Compliance? QA Focus recommendations: Projects should implement appropriate QA procedures Projects may find QA Focus support materials helpful (including case studies) The QA Focus toolkits may be used to help ensure appropriate areas are addressed Compliance issues should be addressed within projects with the expectation that significant discrepancies are brought to attention of funders External auditing of compliance is not felt to be appropriate for the JISC sector / culture

9 QA Focus – Supporting JISC's Digital Library Programmes Applications To Metadata For metadata there is a need to: Clarify the purpose of your metadata (embedded metadata in Web pages won't put you at the top of Google) Have an appropriate architecture for creating and managing your metadata - metadata rots Ensure you have appropriate cataloguing rules Address interoperability with others and not just your own functionality – this is hard Provide appropriate training, documentation, etc. Have appropriate mechanisms for ensuring above work correctly Document the above – for yourselves, for new staff and potentially for 3 rd parties (funders, service providers, …

10 QA Focus – Supporting JISC's Digital Library Programmes Example 1 – ACRONYM Tag Simple example which illustrates several points: Why bother (not supported in IE)? Difference between Acronym and Abbreviation Policy on content (full stops, apostrophes, US vs UK English, …) Error checking – when the metadata is not displayed Solution Simple policy produced. Used Tom Heath's acronym harvester to create automated glossary as (a) value-added service and (b) to check for errors. Policy and approach documented Solution Simple policy produced. Used Tom Heath's acronym harvester to create automated glossary as (a) value-added service and (b) to check for errors. Policy and approach documented

11 QA Focus – Supporting JISC's Digital Library Programmes Example 2 – Web Metadata QA Focus Web site: Uses PHP to process simple metadata (e.g. $author="Brian Kelly", $area ="documents", &keywords="xxx"…) Used for processing ($area used to pull in stylesheet) as well as resource discovery Architecture isn't ideal (backend database better) – but QA is about managing what you've got How do we detect errors? Solution HTML validation detects incorrect PHP syntax errors Visual inspection for correct &area values Keywords & description copied to hard copy document, which allowed for visual checking (and there were errors) Policy documented (shortly) Solution HTML validation detects incorrect PHP syntax errors Visual inspection for correct &area values Keywords & description copied to hard copy document, which allowed for visual checking (and there were errors) Policy documented (shortly)

12 QA Focus – Supporting JISC's Digital Library Programmes What Needs To Be Addressed? You'll need to address: Purpose of metadata Selection of standards, conventions, … Implementation architecture(s) Work flow Project outputs … Note: Some aspects will be already chosen or outside of you control (e.g. FAIR projects will use OAI, …)

13 QA Focus – Supporting JISC's Digital Library Programmes Help Is Available Briefing Documents See Case Studies See Toolkits See You can contribute to the case studies: brief documents covering (a) the project (b) area addressed (c) solution used (d) lessons learnt. You document experiences for sharing with community – and we promote you (at events and on Google-friendly Web site) e.g. Healthier Nation You can contribute to the case studies: brief documents covering (a) the project (b) area addressed (c) solution used (d) lessons learnt. You document experiences for sharing with community – and we promote you (at events and on Google-friendly Web site) e.g. Healthier Nation

14 QA Focus – Supporting JISC's Digital Library Programmes Toolkits Online and paper based toolkits are available Note the toolkits won't solve your problems for you – they are meant as an aid to help you address key areas

15 QA Focus – Supporting JISC's Digital Library Programmes I'm Too Busy! Projects may feel they are too busy to take on this work. However: You're probably doing much of the work already! QA methodology has been designed to be lightweight It is needed in order to provide the interoperability and wide access for which projects are funded Documented QA policies should help ensure the functionality of your deliverables (e.g. when staff leave) There is more likelihood of your project deliverables been deployed into service by others if QA issues have been addressed QA Focus recommends that future JISC programmes require QA to be addressed Projects themselves may prefer the lightweight self- assessment approach than an external audit – could this happen if disasters (such as UKeU) occur?

16 QA Focus – Supporting JISC's Digital Library Programmes Your Feedback Feedback on our approaches are welcome Please complete evaluation forms

17 QA Focus – Supporting JISC's Digital Library Programmes Questions Any questions?