A centre of expertise in digital information management UKOLN is supported by: Pitfalls and blind alleys: perspectives on quality measures Ann Chapman Interoperability Focus

A centre of expertise in digital information management Quality measurement Accuracy – is the data correct? Consistency – is the data consistent? Functional – does the data support users? Currency – is data available when needed

A centre of expertise in digital information management Measuring tape or string? Surveys –Sampling frame, sample sizes –What is being surveyed? Questionnaires –Leading questions, response scales –User perceptions Toolkits –Limitations

A centre of expertise in digital information management Survey samples Sample sizes –Minimum 100 (BNB 1440 over a year) –Split samples, bigger overall sample Sampling frame –Public and academic libraries –Staff and students –Subject areas

A centre of expertise in digital information management Quality = type of record BNB Currency Survey 1980 – 2005 Hit split into record types –Full record from BL –Formerly CIP –CIP Assume records found are correct Sometimes spot error

A centre of expertise in digital information management Errors in data LDLSCP Audit 1994 – 100 records BL, 9 each LD library Checklist of error types –008/260 dont match, spelling, indicators, added entries, consistent data in related records Limited by not seeing items

A centre of expertise in digital information management Edits = errors BLCMP (Talis) study –% of records edited –Analysis of 1310 paired before/after records – but no verification of need to edit –Member libraries submitted a small set of records edited and gave their reasons for editing

A centre of expertise in digital information management Quality = user understanding Cambridge University Study interviews –Random (91 UG, 209 PG/staff) –Users not grouped by area of study Main problems found –Misunderstanding data (jargon) –Difficulty with OPAC labels –Misinterpreting abbreviations

A centre of expertise in digital information management Decide on parameters Cat-Assess Tool ( ) –Record cataloguing policies first –Random sample book/record check BNB Currency Survey –BNB Exclusions policy LDLSCP Audit –Assessment limited by not seeing item –To assess consistency, some related records must be in sample

A centre of expertise in digital information management Re-using data BNB Currency Survey –Acquisitions trends over time –Retention of stock –Comparison with other record suppliers ( ) –Monitor effect of BL policies

A centre of expertise in digital information management So remember … 1.What do you want to measure? 2.Do you already collect data you can use? 3.If not, choose an appropriate method of data collection 4.Write up methodology and results, especially for repeated studies

A centre of expertise in digital information management UKOLN Visit the UKOLN website at: Bibliographic Management Collection Description Focus