1. Tools and Techniques to Clean Up your Database
Meghan Weeks
Systems Librarian
Loyola Marymount University
Southern California IUG Meeting
Oct. 10, 2012

2. Overview Why is it important to have a clean database?
Where to start with a database cleanup project?
How does bad data get into the database?
How do you find bad data?
How do you fix the problems?
How do you prevent issues in the future?

3. Why is it important to have a clean database?
OPAC searching issues
OPAC display issues
Statistics not accurate
Web Management Reports not accurate
Create lists not accurate

4. Where to start? Millennium Database Control Working Group
Formed in Oct. 0f 2010
Biweekly meetings
Members from Acquisitions, Cataloging, Circulation, Media & Reserves, Special Collections, and Systems
Systematically go through each record type and review all of the fixed length and variable length fields
Clean out obsolete fields or values
If needed, create new fields or values
Document what the fields are used for and by whom

5. Document, Document, Document!

6. How does bad data get into the database?
A fixed length field code value is deleted from the table but some records still contain that code
Templates contain invalid entries or no entries

7. How does bad data get into the database?
Valid code with blank label
Valid code but it is a space
Valid codes used incorrectly
Bib location in an item record
Old codes still in the system and being used

8. How do you find bad data?
Patrons and/or staff may report issues that they find in the OPAC
Wrong material type icon
Limiting/Scoping not working as expected
Staff report checkouts giving incorrect due dates
Invalid or wrong patron type, item type, or location
Staff report holds being placed on items that should not circulate
Wrong status or item type

9. How do you find bad data?
Use field statistics for all record types

10. Field Statistics Reports
Look for
Bad Code
Null
` `
Unknown

11. Field Statistics Reports – Fixing Bad Code
Use Create Lists and Rapid Update

12. Field Statistics Reports – Fixing Bad Code
Use Create Lists and Rapid Update

13. Field Statistics – Call Numbers not in SCAT

14. Gap in SCAT Table Review File
After you run field statistics on bib records a report is automatically generated for call numbers not in SCAT table
View report by selecting an empty review file and choose copy
Scroll down toward the bottom of the list and select Bibs: call numbers not in SCAT
Why are call numbers not in the SCAT table?
Adjust the SCAT table or fix records with invalid call numbers

15. How do you find bad data? Web Management Reports
Circulation statistics – stats group issues

16. How do you find bad data? Headings Reports (Cataloging)
Duplicate entries
Item records – barcodes, ISSN, ISBN
Patron records – barcodes
Blind references
Subject authority records
Name authority records
Other headings reports

17. How do you prevent issues in the future?
Edit Preferences (in Login Manager) for each user and make sure invalid text is highlighted

18. How do you prevent issues in the future?
Follow best practices for deleting:
Location codes (107738)
Fund records (100710)
Vendor records (105790)
Check templates for all record types for bad code and missing entries and add prompts
Periodically review load tables
Periodically review fixed length field code values – no blank labels and don’t use a space as a valid code

19. How do you prevent issues in the future?
Check the manual regarding the fixed length field code values that you want to delete or alter.
Source:

20. How do you prevent issues in the future?
Utilize automatic link maintenance or manually run link maintenance every day
Make sure the location mapping table is accurate
Command line interface (aka telnet) - A, L, E
Make sure the MARC Validation table is up-to-date
A, A, S, O ,D ,1
Review scope rules for accuracy
Delete staff accounts when they leave the organization
Periodically review staff authorizations

21. References and Useful Links
Fixing Bad Codes FAQ on CSDirect
Cataloging Clean up Projects
Getting Started with Millennium Statistics
Create and Interpret Reports in Millennium Statistics
Cleaning Up Your Database Presentation by Amy Homick, 2012 IUG Conference, Chicago, IL

22. Thank you! Meghan Weeks Meghan.Weeks@lmu.edu
Questions?
Thank you! Meghan Weeks