1. An analysis of evaluation activities planned in NN/LM outreach subcontracts 2001-2006 Mellanye Lackey 2 nd year Associate Fellow National Library of Medicine UNC-Chapel Hill Health Sciences Library mjlackey@email.unc.edu

2. 2 What I’ll talk about today Gaining background knowledge  Literature review and conversations w. experts  Barriers Project data  Collection methods  Analysis  Results and graphs Opportunities to learn more

3. 3 Barriers to doing outreach evaluation Prior experiences  usefulness  motive  fear negative results Getting buy-in  relationships  cooperation from partners  support from authority  appreciation of evaluation Lack of  time  money  knowledge about evaluation processes - Taut, S. M., & Alkin, M. C. (2003). Program staff perceptions of barriers to evaluation implementation. American Journal of Evaluation. 24, 213-226.

4. 4

5. 5 Data gathering

6. 6 Collecting project data Projects included  Public Health Information Model  Access to Electronic Health Information Subcontract  Consumer Health Initiative Subcontract  Resource Library Outreach Subcontract  Health Professional Information Outreach

7. 7 Project data Funding from $2,000 to $70,000 reporting evaluation plan Reviewed >500 records (1/2 of all NN/LM projects) n = 144

8. 8 Data collection Methods Questionnaire Tally/Log Pre/Post data Interview/Anecdote Evaluation form Focus group Observation/Usability study Approaches Evidence for future use Internal committee Needs assessment Social capital Case study

9. 9

10. 10

11. 11 77822718422916193761

12. 12 $2500- $50,019 $2000- $56,000 $3200- $50,000 $2500- $50,000 $17,973$40,000

13. 13 Limits of the data Records created at project inception Data sometimes rich, more often brief  How many participants? Focus groups? How much time?  Terms vague (eg. evaluation form) Growing emphasis on evaluation with Outreach Evaluation Resource Center (OERC)

14. 14 Further research ideas Compare evaluation planned with final report Get more specifics about each plan Examine formative vs. summative Collect information about “approaches” How get more data?

15. 15 Conclusions At all funding levels:  Intent to evaluate  Variety of methods Opportunities to learn more  Bibliography of resources http://www.hsl.unc.edu/AboutLib/Staff/LiaisonWebPages/PHliaison.cfm  Outreach Evaluation Resource Center http://nnlm.gov/evaluation/index.html