1. CLEVER EVALUATION Nikki Dettmar Evaluation Librarian NN/LM Outreach Evaluation Resource Center Presentation to the NN/LM Pacific Southwest Region February 19, 2014

2. Why don’t we exercise?

3. Evaluation is like exercise It’s a good idea…and you’ll start doing it as soon as you have time

4. Advice for doing evaluation is the same as for exercise Plan wellSet realistic goals Use efficient techniques

5. These strategies will get you across the finish line

6. Plan for Evaluation

7. Know how you will use evaluation Needs/ Community ProcessResults and Outcomes Star t Finish

8. Needs and community assessment can help you… Explore needs Identify resources and opportunities Use Design services/programs Establish baseline info Get stakeholder buy-in

9. Process assessment keeps you on the right track Monitor deliverables Assess quality Use Make midproject adjustments Report lessons learned

10. Outcomes evaluation is used to show results Report on planned outcomes Identify unplanned outcomes Use Decision-making Advocacy

11. Time Saver #1: Time saver: Community/needs assessment: Don’t assess what you (or someone else) already knows

12. Explore the collective knowledge of external experts http://phpartners.org http://phpartners.org/health_stats.html

13. For primary (“local”) information, think “shoestring” methods Key informant interviews Focus group interviews Short surveys Rapid response methods

14. Time Saver #2 For process and outcomes assessment, use a project plan

15. Think Through Activities, Outcomes and Indicators Activities If we do these things Outcomes We will get these results Indicators We will “see” this if…

16. Think through activities, outcomes, and indicators Activities Indicators: Deliverables Job well done Outcomes Indicators: Evidence of results

17. Diffusion of Innovations theory: People adopt new innovations in phases KnowledgePersuasionDecision Implemen- tation Confirm- ation

18. Design measures for the behavioral changes you are targeting KnowledgePersuasion Decision Implemen- tation Confirm- ation Awareness Favorable Attitude Trial useApplication Behavioral-level Outcomes

19. Think “small ball” POWER BALL: SMALL BALL: Large number of VS. Winning games runs in few by scoring inningssingle runs in multiple innings

20. Communities adopt innovations in waves Source: http://upload.wikimedia.org/wikipedia/en/4/ 45/DiffusionOfInnovation.png

21. Use different strategies for different parts of the adoption process KnowledgePersuasionDecision Implement- ation Confir- ation Awareness Favorable Attitude AdoptionApplication Behavioral-level Attitudes Use to help others Promote to others

22. Example 1: Train-the-trainer programs Who are you targeting? Opinion leaders What are the planned outcomes? Use for self Use for others Promotion to others How would you measure them? Post-training attitude measures Stories of use

23. Other “clever evaluation” time savers

24. Interviews and focus groups can be networking opportunities

25. …And marketing opportunities

26. Evaluation techniques can support learning

27. Use tests to reinforce learning or practice skills QUIZ 1.What is the “important warning” about the drug Warfarin? _______________ 2.What usually are the first warning signs of pulmonary hypertension?_________ 3.Name one health news article from November 5, 2008 _______________

28. Pose questions to help participants process information 1.“List one example of information you found today that helped you with a health concern” 2.“List two topics you plan to investigate using MedlinePlus” 3.“List 2 or 3 health topics that you plan to research at home this week using MedlinePlus”

29. Work with others as a team

30. Don’t forget to ask for help with analysis, too

31. Free Support Resources Planning and Evaluating Health Information Outreach Projects Available online http://nnlm.gov/evaluation/guides.html http://nnlm.gov/evaluation/guides.html Want a print copy of the set? Email me! OERC Tools and Resources http://guides.nnlm.gov/oerc/tools http://guides.nnlm.gov/oerc/tools

32. Nikki Dettmar, Evaluation Librarian University of Washington, Seattle 206-543-3409 snydern@uw.edu OERC Contact Information (nnlm.gov/evaluation)nnlm.gov/evaluation