National Network of Libraries of Medicine Outreach Evaluation Resource Center Planning Outcomes-Based Outreach Programs.

Slides:



Advertisements
Similar presentations
Donald T. Simeon Caribbean Health Research Council
Advertisements

Using Data to Measure and Report Program Impact Anne Basham, MFA MEMconsultants.
How to Evaluate Your Health Literacy Project Jill Lucht, MS Project Director, Center for Health Policy
CLEVER EVALUATION Nikki Dettmar Evaluation Librarian NN/LM Outreach Evaluation Resource Center Presentation to the NN/LM Pacific Southwest Region February.
Program Evaluation and Measurement Janet Myers. Objectives for today… To define and explain concepts and terms used in program evaluation. To understand.
Creating an Evaluation Plan Freya Bradford, Senior Consultant NorthSky Nonprofit Network May 23, 2013.
1 Theory of Change Chesapeake Bay Funders Network Program Evaluation Training Workshop OMG Center for Collaborative Learning January 9-10, 2008.
Outreach Evaluation Series: Community Assessment Susan Barnes and Alan Carr National Network of Libraries of Medicine Outreach Evaluation Resource Center.
Focusing Your Evaluation Activities Chapter Four cont…
Logic Modeling for Success Dr Kathryn Wehrmann Illinois State University.
Logic Models. A logic model is your program ROAD MAP. Where are you trying to go? How are you trying to get there? What will tell you that you’ve arrived?
Evaluation 2.0: Trends, New Ideas, Cool Tools Cindy Olney NN/LM Outreach Evaluation Resource Center January 22, 2015.
Improving Patient Outcomes Through Effective Teaching The Teach Back Method.
Continuous Quality Improvement (CQI): Part 2
Community Planning Training 1-1. Community Plan Implementation Training 1- Community Planning Training 1-3.
1 Classroom-Based Research: How to Be a Researcher in Your Classroom Basic Skills Initiative Teaching and Learning Workshop October 2009 Darla M. Cooper.
Measuring for Success Module Nine Instructions:
How to Develop the Right Research Questions for Program Evaluation
Evaluation 101 Everything You Need to Know to Get Started Evaluating Informal Science Education Media Everything You Need to Know to Get Started Evaluating.
How to Focus and Measure Outcomes Katherine Webb-Martinez Central Coast and South Region All Staff Conference April 23, 2008.
Plan, Evaluate and Improve Your Health Literacy Interventions and Activities Sabrina Kurtz-Rossi, M.Ed. Health Literacy Consultant April 10, 2013.
1 Qualitative Evaluation Terms Coding/categorization The process of condensing qualitative data through the identification of common themes. Data Matrix.
CENTER FOR NONPROFIT EXCELLENCE EVALUATION WORKSHOP SERIES Session IV: Analyzing and Reporting Your Data Presenters: Patty Emord.
ISTEP: Technology Field Research in Developing Communities Instructor: M. Bernardine Dias CAs: Sarah Belousov and Ermine Teves Spring 2009.
Darren A. DeWalt, MD, MPH Division of General Internal Medicine Maihan B. Vu, Dr.PH, MPH Center for Health Promotion and Disease Prevention University.
Washington State Library OBE Retreat October 20, 2006 Matthew Saxton & Eric Meyers.
Measuring the Value of Your Volunteer Efforts Nikki Russell Volunteer Initiatives Manager United Way of King County.
Once Upon a Time Evaluation Reports That Get Attention Cynthia Olney NN/LM Outreach Evaluation Research Center February 21, 2013.
The Evaluation Plan.
CONFERENCE EVALUATION PLANNING Good planning is essential ! (‘’fail to plan is plan to fail’’)
SUPPORT FOR YOUR STUDENT EQUITY PLAN Presented by the Institute for Evidence-Based Change October 10th, 2014.
Innovative Teaching and Learning MOVING FROM THEORY TO ACTION 6 th June 2012.
© 2015 Cengage Learning. All Rights Reserved. May not be scanned, copied or duplicated, or posted to a publicly accessible website, in whole or in part.
Food Safety Professional Development for Early Childhood Educators Evaluation Plan.
Reflect and Revise: Evaluative Thinking for Program Success Tom DeCaigny and Leah Goldstein Moses.
Helen Tannenbaum Outreach and Communications Coordinator NN/LM Middle Atlantic Region MAR Update.
Overview of the Plain Talk Data Collection System Sarabeth Shreffler, MPH, CHES Program Officer, Plain Talk Program Public/Private Ventures.
Grant Writing Basics. Topics of This Session Matching funding to your objective Telling your story Writing the budget.
2013 NEO Program Monitoring & Evaluation Framework.
© 2012 Cengage Learning. All Rights Reserved. This edition is intended for use outside of the U.S. only, with content that may be different from the U.S.
Monitoring and Evaluation of GeSCI’s Activities GeSCI Team Meeting 5-6 Dec 2007.
Indicators for ACSM.
1 © 2010 Regents of the University of Minnesota. All rights reserved. 11 Logic Models A ROADMAP TO SUCCESS.
Big6 Overview Big6™ Trainers Program McDowell County Schools.
Take Charge of Change MASBO Strategic Roadmap Update November 15th, 2013.
Evaluation Workshop Self evaluation – some workable ideas and approaches.
HIL Research Project Evaluation Plan and Timeline April 15, 2008 A project of the Medical Library Association working with the National Library of Medicine.
Monitoring and Evaluation
Measuring for Success Module Nine. Reflecting on the Previous Session What was most useful? What progress have you made? Any comments or questions?
Program Assessment. Before you get started Need to determine if the program can be evaluated or should be evaluated. Is the answer already available?
1 Evaluation of Patient-Centered Medical Home (PCMH) Initiatives Meredith B. Rosenthal, PhD February 24, 2009.
National Center on Health: Four-Year Evaluation of Outcomes Head Start Oral Health Project *Isman, BA; Geurink, K; Holt, K; Landrum, M; Goodman, H.
Adrienne DiTommaso, MPA, CNCS Office of Research and Evaluation
TDRp Implementation Challenges David Vance, Executive Director Peggy Parskey, Assistant Director October 23, 2014.
1 The project is financed from the European Union funds within the framework of Erasmus+, Key Action 2: Cooperation for innovation and the exchange of.
Making it Count! Program Evaluation For Youth-Led Initiatives.
Adding evaluation to your plan and next steps: proposal Webinar 4 of the series Mapping an Outreach Project: Start with Information; End with a Plan The.
ADRCs Do What? Using Logic Models to Document and Improve ADRC Outcomes Glenn M. Landers.
Applied Methodologies in PHI Session 5: Evaluation Kath Roberts (NOO/EMPHO)
Getting to Know a New Community through Community Assessment National Network of Libraries of Medicine Outreach Evaluation Resource Center.
SUPPORT FOR YOUR STUDENT EQUITY PLAN
Logic Models How to Integrate Data Collection into your Everyday Work.
Designing Effective Evaluation Strategies for Outreach Programs
An agency of the Office of the Secretary of Education and the Arts
CATHCA National Conference 2018
Regulated Health Professions Network Evaluation Framework
Changing the Game The Logic Model
NICE has many methods and processes
Presentation transcript:

National Network of Libraries of Medicine Outreach Evaluation Resource Center Planning Outcomes-Based Outreach Programs

You got to be very careful if you don’t know where you’re going because you might not get there -- Yogi Berra

Why talk about outcomes in the planning phase? What gets measured gets done If you can’t recognize failure, you can’t correct it If you cannot see success, you can’t reward it (and may be rewarding failure) If you do not measure results, you cannot see success If you can demonstrate results, you can win public support From Reinventing Government

A plan helps you move in the right direction. Evaluation keeps your eyes on the road

By the end of this workshop you will know how to Use logic models to plan programs around desired results (outcomes) Shape objectives from outcomes Develop evaluation plans based on logic models

The Logic Model Creating the map

Begin with the results you’re seeking

Figure out what you have to do

Then identify what you need

Activities are more under your control than outcomes

Here are examples of the items for each column

Goal: Have a good summer vacation

Vacation: Output versus Outcome

Library Project: Output Focus

Library Project: Outcome Focus

There are two general categories of outcomes

Examples of system-level change

People change in stages

Kirkpatrick’s Model of Program Evaluation is popular for evaluating training programs

Kirkpatrick’s Model can be useful in completing the Logic Model

Last step: Get Feedback from stakeholders and key informants

How could the logic model help plan proposals? Logic ModelProposal InputsBudget ActivitiesStrategies OutputsDeliverables OutcomesResults & Evaluation InfluencesSupport & Barriers AssumptionsReviewers’ questions

Use the logic model to write final reports Logic ModelReport InputsResources you acquired/used ActivitiesWhat you did OutputsDeliverables Outcomes What you accomplished Influences Lessons learned Assumptions Lessons learned

Making outcomes measurable: Writing objectives

Build objectives around your outcomes

Indicators are the observable signs of the outcome Outcome Parents will increase their ability to find online health information Indicator (measurable) Parents will report they were able to use MedlinePlus to locate information on at least one health topic that was important to them

The target sets the criteria for success Indicator (measurable) Parents will report being able to use MedlinePlus to locate information on at least one health topic that was important to them Target (criterion for success) 70% of participants

Time frame tells you when you are going to check on the indicator

Now you can write the outcome objective for an outcome Outcome Parents will increase their ability to find online health information Outcome Objective By the end of the training session, 70% of parents will report being able to use MedlinePlus to locate information on at least one health topic that was important to them

Example Objective: Awareness Outcome The OERC will improve access to information about emerging evaluation trends among NN/LM members Outcome Objective By the May 2015, All 8 regional medical libraries will have re-posted OERC blog articles in their Newsletters or blog

Objective Example: Impact Outcome Clinicians will recommend health information to their patients Outcome Objective One month after training, 60% of clinicians will report providing at least one patient with Information about MedlinePlus

Discussion: What indicators could you use? Increased use of health resources Indicator: ____________________ More access to health resources Indicator: ____________________ More cost effectiveness Indicator: ____________________ Increased efficiency Indicator: ____________________

Webinar 4: Adding Evaluation to Your Plan

What your stakeholders want to know  Did you get there?  Did you do everything you planned?  How Did you get there?  Did you do everything you planned?  What unexpected things happened?  Would you go there again?  What advice do you have for others?  What would you do differently next time?  What nexpected things happened?  Would you go there again? What advice do you have for others? What would you do differently next time?

Focus on “Rapid” methods Short surveys Site visits Observations Key informant and focus group interviews Existing data review

Use mixed methods Quantitative methods for numbers Qualitative methods for text or images

Source and Method Where we find dataHow we collect data

Process evaluation: Keeping your eyes on the road

Process assessment is developed for planned activities

Documenting process

How to describe progress

Add check-ins with key players

Comments and feedback allow you to assess quality of your program

Outcomes Assessment: Did you make it and was it worth it?

Outcomes assessment is guided by the “intended results’ columns

How do you evaluate your outcome objective? By the end of the training session, 70% of parents will report being able to use MedlinePlus to locate information on at least one health topic that was important to them DATA SOURCE Participants EVALUATION METHOD Session Evaluation Form DATA COLLECTION TIMING At the end of each training session

Identify Source, Method, Timing By the May 2015, All 8 regional medical libraries will have re-posted OERC blog articles in their newsletters or blog DATA SOURCE NN/LM websites EVALUATION METHOD Review of blogs and newsletters DATA COLLECTION TIMING At the end of each quarter

Example: ¡VIVA! Outreach Project Objectives: When talking with patients, physicians and physicians assistants will increase use of : Plain language Open-ended questions Teach back Patient health information that matches language and literacy needs MedlinePlus as a patient information source

Example: The STISD Health Literacy Internship Project

Most Significant Change Method

Basic MSC Questions What is the most significant change you have experienced since________ Why was this change so significant to you? What different has it made already? What difference will it make in the future?

NN/LM Outreach Evaluation Resource Center Visit us here: Website: Blog: Facebook: Cindy Olney Acting Assistant Director Karen Vargas Evaluation Specialist