CLEVER EVALUATION Nikki Dettmar Evaluation Librarian NN/LM Outreach Evaluation Resource Center Presentation to the NN/LM Pacific Southwest Region February 19, 2014

Why don’t we exercise?

Evaluation is like exercise It’s a good idea…and you’ll start doing it as soon as you have time

Advice for doing evaluation is the same as for exercise Plan wellSet realistic goals Use efficient techniques

These strategies will get you across the finish line

Plan for Evaluation

Know how you will use evaluation Needs/ Community ProcessResults and Outcomes Star t Finish

Needs and community assessment can help you… Explore needs Identify resources and opportunities Use Design services/programs Establish baseline info Get stakeholder buy-in

Process assessment keeps you on the right track Monitor deliverables Assess quality Use Make midproject adjustments Report lessons learned

Outcomes evaluation is used to show results Report on planned outcomes Identify unplanned outcomes Use Decision-making Advocacy

Time Saver #1: Time saver: Community/needs assessment: Don’t assess what you (or someone else) already knows

Explore the collective knowledge of external experts

For primary (“local”) information, think “shoestring” methods Key informant interviews Focus group interviews Short surveys Rapid response methods

Time Saver #2 For process and outcomes assessment, use a project plan

Think Through Activities, Outcomes and Indicators Activities If we do these things Outcomes We will get these results Indicators We will “see” this if…

Think through activities, outcomes, and indicators Activities Indicators: Deliverables Job well done Outcomes Indicators: Evidence of results

Diffusion of Innovations theory: People adopt new innovations in phases KnowledgePersuasionDecision Implemen- tation Confirm- ation

Design measures for the behavioral changes you are targeting KnowledgePersuasion Decision Implemen- tation Confirm- ation Awareness Favorable Attitude Trial useApplication Behavioral-level Outcomes

Think “small ball” POWER BALL: SMALL BALL: Large number of VS. Winning games runs in few by scoring inningssingle runs in multiple innings

Communities adopt innovations in waves Source: 45/DiffusionOfInnovation.png

Use different strategies for different parts of the adoption process KnowledgePersuasionDecision Implement- ation Confir- ation Awareness Favorable Attitude AdoptionApplication Behavioral-level Attitudes Use to help others Promote to others

Example 1: Train-the-trainer programs Who are you targeting? Opinion leaders What are the planned outcomes? Use for self Use for others Promotion to others How would you measure them? Post-training attitude measures Stories of use

Other “clever evaluation” time savers

Interviews and focus groups can be networking opportunities

…And marketing opportunities

Evaluation techniques can support learning

Use tests to reinforce learning or practice skills QUIZ 1.What is the “important warning” about the drug Warfarin? _______________ 2.What usually are the first warning signs of pulmonary hypertension?_________ 3.Name one health news article from November 5, 2008 _______________

Pose questions to help participants process information 1.“List one example of information you found today that helped you with a health concern” 2.“List two topics you plan to investigate using MedlinePlus” 3.“List 2 or 3 health topics that you plan to research at home this week using MedlinePlus”

Work with others as a team

Don’t forget to ask for help with analysis, too

Free Support Resources Planning and Evaluating Health Information Outreach Projects Available online Want a print copy of the set? me! OERC Tools and Resources

Nikki Dettmar, Evaluation Librarian University of Washington, Seattle OERC Contact Information (nnlm.gov/evaluation)nnlm.gov/evaluation