IIPC GA Curator Tools Fair May 2014 WEB CURATOR TOOL Nicola Bingham Web Archivist

2 Introduction Jointly developed by BL and NLZ 2006 under the auspices of the IIPC WCT manages the selective web harvesting process Designed for use in libraries by non-technical users Open source Uses the Heritrix web crawler

3 What it does and doesn’t do. Appraisal and selection: choosing websites for capture. –Subject specialists, curators, external agencies –BL uses a selection permission tool plugged into WCT Metadata/Description –Basic Dublin Core Metadata –Titles, description, subject and collection tagging Scoping and Data Capture –Scheduling –Crawl parameters, e.g. path depth, size of download QA and Analysis –Heritrix log files –Browse tools –Recommendations based on indicators

4 What it does and doesn’t do continued.. Storage and Organisation –WARC files created in WCT –Passed out of WCT for indexing and long term storage Access/Use/Reuse –Wayback is plugged in as the access tool –Harvested sites can be viewed within the tool Risk Management –Harvest Authorisation module, rights metadata –Records the outcome of publisher communications –Control the display of Targets

5 Development Latest version available now. UI new features and improvements (x 17) including… –Date pickers for date fields –Scheduling heat map –Harvest optimisation Bug Fixes (x11) Development related e.g., –No longer need to install Apache Tomcat server or database etc NLNZ budgeted NZD 50,000 for Open development process up to all WCT users. –WCT pages –Wiki (Code, Support, mailing lists, bug tracker)

6 Thank-you. UK Web Archive