INCREASING IMPACT Understanding Measurements of Impact in Evaluation June 2013 Jennifer Farley.

Slides:

Advertisements

Similar presentations

Program Evaluation: What is it?

Advertisements

Quality Improve program impact by increasing use of evidence-based practices Relevance Increase program relevance and diversity of constituents served.

Project Management.

Focusing an Evaluation Ben Silliman, Youth Development Specialist NC 4-H Youth Development.

Program Evaluation. Overview and Discussion of: Objectives of evaluation Process evaluation Outcome evaluation Indicators & Measures Small group discussions.

Managing the Statutory Requirements for Assessment April 2011.

ICTN PROJECT WEBINAR SERIES COMMUNITY HEALTH STUDENT PLACEMENT PROGRAM Presented By: Robyn Stringer, Judith Foley-Chell and Helen Johnson.

Evaluation Capacity Building Identifying and Addressing the Fields Needs.

Evaluating the Tikkun Middot Project Tobin Belzer PhD October 28, 2013.

Through Instructional Rounds

Key Stage 3 National Strategy Standards and assessment: session 2.

Overview M&E Capacity Strengthening Workshop, Maputo 19 and 20 September 2011.

What can outcomes based planning and evaluation do for you? Elizabeth Kryder-Reid, Director, Museum Studies, IUPUI and Principal Investigator, Shaping.

Overview of Mission Statement & Organization. Development Communication Division External Affairs Vice Presidency DevComm Vision To put communication.

Presentation at The Conference for Family Literacy Louisville, Kentucky By Apter & O’Connor Associates April 2013 Evaluating Our Coalition: Are We Making.

MODULE 8: PROJECT TRACKING AND EVALUATION

RMS – a collaborative approach Presentation Lyn Dare & Stephen Larmour Authorisation & Audit Comcare.

The Fundamentals of Evaluating Prevention Programs Colleen Ryan, Canada’s Drug Strategy Secretariat, Health Canada.

Program Evaluation. Lecture Overview  Program evaluation and program development  Logic of program evaluation (Program theory)  Four-Step Model  Comprehensive.

1 Change Management Overview. 2 Below are some facts based on research conducted on managing the resistance to change: 500 executives said that resistance.

Measuring for Success NCHER Legislative Conference Sophie Walker September 26, 2013.

Problem Based Learning and Teacher Technology Training.

The Lumina Center Grantseeking Workshop Series Presents Outcomes & Evaluations April 20, 2006.

Assessing Financial Education: A Practitioner’s Guide December 2010.

How to Develop the Right Research Questions for Program Evaluation

Product Evaluation the outcome phase. Do the magic bullets work? How do you know when an innovative educational program has “worked”? How do you know.

ADVANCED LEADERSHIP DEVELOPMENT FOR SUPERVISORS (ALDS) PROGRAM YEAR 1 IMPLEMENTATION MARCH 2010 – FEBRUARY 2011 PILOT PROGRAM.

Eight Steps to Improving Career Services in Schools, Colleges, and Agencies James P. Sampson, Jr. National Career Development Association Global Conference.

Program Evaluation Tool Overview Library Media Leadership Council / Instructional Technology Leaders October 2, 2014.

Too expensive Too complicated Too time consuming.

Implementing Social Accounting OISE/UT Social Accounting Certificate Course April 2004 B.J. Richmond Faculty of Education, York U.

Monitoring and Evaluation in MCH Programs and Projects MCH in Developing Countries Feb 10, 2011.

1 Introduction to Evaluating the Minnesota Demonstration Program Paint Product Stewardship Initiative September 19, 2007 Seattle, WA Matt Keene, Evaluation.

Fundamentals of Evaluation for Public Health Programs ROBERT FOLEY, M.ED. NIHB TRIBAL PUBLIC HEALTH SUMMIT MARCH 31,

Military Family Services Program Participant Survey Training Presentation.

Outcome Based Evaluation for Digital Library Projects and Services

Cathy Burack and Alan Melchior The Center for Youth and Communities The Heller School for Social Policy and Management, Brandeis University Your Program.

Effective Instructional Feedback Mike Miles July 2009.

Juggling the Program Management Ball 1. One day thou art an extension educator… The next day thou art a county director or district extension director…how.

Purposes of Evaluation Why evaluate? Accountability: Justify trust of stakeholders (funders, parents, citizens) by showing program results (Summative)

Autism Team Training in South Dakota Presented by Brittany Schmidt, MA-CCC/SLP Center for Disabilities Autism Spectrum Disorders Program

RPPS Education Development Process Debbie Bender.

1. Housekeeping Items June 8 th and 9 th put on calendar for 2 nd round of Iowa Core ***Shenandoah participants*** Module 6 training on March 24 th will.

A working team of professionals that are committed to ensure student achievement as life-long learners of the 21 st Century.

1 VSkills Adrian Murtagh Head of Development. Our role within the, ‘Volunteering - Way to Employment’ project To share our knowledge, skills and expertise.

704: Conducting Business in Fiscally Challenging Times: Strategies and Tools to Get There PCYA Leadership Academy Presentation March 28, 2012.

Professional Development Implementation Team January 28, 2013.

Military Family Services Program Participant Survey Briefing Notes.

Community Planning Training 5- Community Planning Training 5-1.

Program Evaluation.

DESIGNING, IMPLEMENTING, AND EVALUATING CAREER DEVELOPMENT PROGRAMS AND SERVICES Career Development Interventions in the 21 st Century 4 th Edition Spencer.

Idaho Energy Plan Proposal (RFI) Energy Policy Institute (EPI)

Strategies for Knowledge Management Success SCP Best Practices Showcase March 18, 2004.

Monitoring and Evaluation in MCH Programs and Projects MCH in Developing Countries Feb 24, 2009.

Russell Frank, Ph.D. Director II Assessment and Accountability Services Mike Barney Director II Instructional Services Getting LTELs Ready for College.

Adrienne DiTommaso, MPA, CNCS Office of Research and Evaluation

R 0 G125 B 177 R 78 G 47 B 145 R 185 G 50 B 147 R 245 G132 B 107 R 255 G234 B 83 R 123 G193 B 67 R149 G169 B Goal Setting Overview.

Student Learning Objectives 1 SCEE Summit Student Learning Objectives District Professional Development is the Key 2.

Planning Evaluation Setting the Course Source: Thompson & McClintock (1998) Demonstrating your program’s worth. Atlanta: CDC/National Center for Injury.

Program Evaluation Principles and Applications PAS 2010.

Monitoring and Evaluation in MCH Programs and Projects MCH in Developing Countries Feb 9, 2012.

A Professional Development Series from the CDC’s Division of Population Health School Health Branch Professional Development 101: The Basics – Part 1.

So You Think You’ve Made a Change? Developing Indicators and Selecting Measurement Tools Chad Higgins, Ph.D. Allison Nichols, Ed.D.

Business Leadership Network: Program Evaluation in Public Health March 28, 2016.

Overview of Education in Health Care

Publisher to insert cover image here Chapter 9

Building Partnerships: How the Office of Assessment and Accreditation Can Help You and Your Program Be Successful.

مدل زنجیره ای در برنامه های سلامت

Rubrics: a good tool for engaging thoughtful learning

Presentation transcript:

INCREASING IMPACT Understanding Measurements of Impact in Evaluation June 2013 Jennifer Farley

Overview Why evaluate? Evaluation: Types, Continuous Improvement and Neglect Outputs vs. Impact Measures of Impact Is impact good enough? When it’s not good enough Maximizing Evaluation Questions

Why Evaluate? Ensures Quality Increases Knowledge Prioritize Resources Accountability Demonstrate need and effectiveness Marketable (Russ-Eft & Preskill)

Types of Evaluation Developmental Formative Summative Monitoring and Auditing Outcome Evaluation Impact Evaluation Performance Measurement (Russ-Eft & Preskill)

Continuous Improvement Evaluation

Top 10 Reasons Evaluation is Neglected 1. Misunderstanding the purpose and role of evaluation 2. Fear 3. Lack of Skills (Real or Perceived) 4. No one has asked for it 5. It is not used 6. Time-consuming 7. Perceived costs vs. Perceived benefit 8. Leaders already know what works 9. Prior experience 10. Lack of Value (Russ-Eft & Preskill)

Outputs and Impact

Measures of Impact Measures of Change Attitude, Knowledge and Behavior Comparison Pre- and Post-program Calibration Is the primary impact what is measured? Is the program driving the measure, or is the measure driving the program?

Is the Impact Good Enough? Goals Program established goals Funder established goals Comparison Population level data Are the target and general population the same? The continuum of evaluation and research

When Impact isn’t “Good Enough” Re-calibrate Measures What is impact? Participant focus groups What best measures impact? Refine measurement tools Staff Training Re-Pilot surveys Incorporate Measures of Program Quality Look at HOW programs and services are offered Explore field-specific measures of quality Youth Program Quality Assessment Quality Rating System

Results: Maximizing Evaluation What is the purpose of the evaluation? Funder Required? Is there a requirement to “Act” on the results? How are the results communicated to all stakeholders? How are results used in planning?

Questions?

Contact Information Jennifer Farley Censeo Solutions

Works Cited Russ-Eft, D., & Preskill, H. (2009). Evaluation In Organizations: A systematic approach to enhancing learning, performance, and change. New York: Basic Books.