Evaluating your Repository Institutional Repository Workshop 1 – 3 April 2009 Presented by ina.smith@up.ac.za
Evaluation “The process of determining whether a product meets specified criteria.” “An assessment of the project's success in meeting its intended outcomes; usually includes a formative evaluation and summative evaluation.” Source: http://www.google.co.za/search?hl=en&rlz=1T4ADBR_enZA275ZA276&defl=en&q=define:Evaluation&sa=X&oi=glossary_definition&ct=title
Formative Evaluation “A type of process evaluation of new programs or services that focus on collecting data on program operations so that needed changes or modifications can be made to the program in the early stages.” Source: www.ojp.usdoj.gov/BJA/evaluation/glossary/glossary_f.htm “Usability analyses carried out early and throughout system development with the goal of guiding design.” Source: ei.cs.vt.edu/~cs5714/glossary.html
Summative Evaluation “An evaluation carried out after a program has been fully developed; the purpose of a summative evaluation is to see whether the program has achieved the objectives set for it.” Source:srmdc.net/glossary.htm “Usability analyses carried out at the end or at milestones during system development with the goal of assessing how well the system has met its usability objectives.” Source:ei.cs.vt.edu/~cs5714/glossary.html
Tools for evaluating your IR Interviews Surveys or checklists Dialogue with participants
Evaluation Workflow Implementation of Software on Server Redeploys software Evaluation Team conducts evaluation Quality Assurance Server Developmental Server Production Server Feedback to Evaluation Team Report to IR IT Manager IR IT Manager resolves issues/ add changes
Evaluation Instrument & Test Report See Manual Part 10 – to test DSpace functionalities, add changes/ improvements
Formal Evaluation Instruments DRIVER http://validator.driver.research-infrastructures.eu/validatorWeb/ TRAC http://www.crl.edu/content.asp?l1=13&l2=58&l3=162&l4=91 Repository Explorer http://re.cs.uct.ac.za/ DRAMBORA http://www.repositoryaudit.eu/
Questions? ina.smith@up.ac.za