A Hierarchy for Targeting Outcomes and Evaluating Their Achievement S. Kay Rockwell Professor and Evaluation Specialist Agricultural.

Slides:

Advertisements

Similar presentations

Program Evaluation How to Effectively Evaluate Your Program Carol Pilcher Department of Entomology Iowa State University.

Advertisements

Program Design Roger A. Rennekamp, Ph.D. Extension Professor and Specialist in Program and Staff Development Department of Community and Leadership Development.

Presentation at The Conference for Family Literacy Louisville, Kentucky By Apter & O’Connor Associates April 2013 Evaluating Our Coalition: Are We Making.

Using Data to Measure and Report Program Impact Anne Basham, MFA MEMconsultants.

Steve Meier. What is Strategic Planning Determines Where an organization is going over the next year or more, How it's going to get there How it'll know.

How to Evaluate Your Health Literacy Project Jill Lucht, MS Project Director, Center for Health Policy

Corry Bregendahl Leopold Center for Sustainable Agriculture Ames, Iowa.

Program Evaluation and Measurement Janet Myers. Objectives for today… To define and explain concepts and terms used in program evaluation. To understand.

Dennis McBride, Ph.D. The Washington Institute (253) Goal Driven Logic Models.

Extension Program Evaluation Michigan Planning and Reporting System (MI PRS) Winter 2011Training.

Evaluation Planning & Outcome Measurement Beyond the Logic Model 2010 AAEA In-Service, Maricopa Agricultural Center Al Fournier, Dan McDonald.

The Lumina Center Grantseeking Workshop Series Presents Outcomes & Evaluations April 20, 2006.

Programming Techniques and Skills for Advisory Leaders Ralph Prince and Roger Rennekamp, Ph.D. University of Kentucky Cooperative Extension Service.

Who was there and how many were there? Did they learning something? Will they change behavior? Inputs: Time, Money, Materials Numbers Reached, Frequency.

Measuring Success & Impact: Challenges & Opportunities for STEM Early Learning Programs Tiffany R. Lee University of Colorado Boulder University of Washington.

Evaluation. Practical Evaluation Michael Quinn Patton.

Performance Management Upul Abeyrathne, Dept. of Economics, University of Ruhuna, Matara.

Defining & Reporting Our Successes Outputs, Outcomes and Impacts.

Molly Chamberlin, Ph.D. Indiana Youth Institute

Helping Non-Profit Organizations Understand The Value of Their Work Copyright Postrain Consulting Services 2009.

How to Focus and Measure Outcomes Katherine Webb-Martinez Central Coast and South Region All Staff Conference April 23, 2008.

Program Planning & Evaluation Begin with the End in Mind Dr. Dallas L. Holmes Specialist Institutional Research Utah State University Extension.

Logic Models Handout 1.

Program Evaluation and Logic Models

WRITING EFFECTIVE GRANT PROPOSALS With an Eye toward Performance and Evaluation Issues.

1 Introduction to Evaluating the Minnesota Demonstration Program Paint Product Stewardship Initiative September 19, 2007 Seattle, WA Matt Keene, Evaluation.

Fundamentals of Evaluation for Public Health Programs ROBERT FOLEY, M.ED. NIHB TRIBAL PUBLIC HEALTH SUMMIT MARCH 31,

The Targeting Outcomes of Programs (TOP) framework.

Model mod·el noun \mä-d ə l\ 1:Structural design. 2:A usually miniature representation of something; a pattern of something to be made. 3:An example for.

Outcome Based Evaluation for Digital Library Projects and Services

EVALUATION RESEARCH. Evaluation Research  How do we begin?  What are the different types of evaluation research?  How do these different types fit.

Cathy Burack and Alan Melchior The Center for Youth and Communities The Heller School for Social Policy and Management, Brandeis University Your Program.

Understanding monitoring and evaluation FACTSHEET 8 Food and Garden Organics Best Practice Collection Manual.

Juggling the Program Management Ball 1. One day thou art an extension educator… The next day thou art a county director or district extension director…how.

 How do we begin?  What are the different types of evaluation research?  How do these different types fit together?  What purpose do they serve?

LCES Advisory Leadership System… The role of advisory leadership councils in Extension’s programming process.

Futuring the Key to NC Success Pat Sobrero NC Urban Extension Summit May 11, 2005.

Julie R. Morales Butler Institute for Families University of Denver.

Prepared by the North Dakota State Data Center July HNDECA and ECCS Evaluation Dr. Richard Rathge Professor and Director North Dakota State Data.

Community Business Leaders of Tomorrow Kyle Merten Jeanie Long “Leading towards a better tomorrow.”

Award, Certificate and Diploma in Strategic Management Certificate and Diploma in Executive Leadership ILM Level 7.

Center for Leadership Development Guarantee The Money: Making Your Case Through Program Evaluation.

Dr. Shulagna Sarkar Assistant Professor , Jt. Coordinator – PGDM – HRM

Family Resource and Youth Services Centers: Action Component Plan.

“A Truthful Evaluation Of Yourself Gives Feedback For Growth and Success” Brenda Johnson Padgett Brenda Johnson Padgett.

Mapping the logic behind your programming Primary Prevention Institute

Assessment of Student Learning in General Education AAHE/NCA 2003 Assessment Workshop Omaha, Nebraska ● June 2003.

Community Planning 101 Disability Preparedness Summit Nebraska Volunteer Service Commission Laurie Barger Sutter November 5, 2007.

Community Investment Reviewer Training United Way of Greater New Haven March 2010.

ORGANIZATIONAL EFFECTIVENESS

Monitoring and Evaluation in MCH Programs and Projects MCH in Developing Countries Feb 9, 2012.

Using Logic Models to Create Effective Programs

Dubai Statistics Center practices in Human Resources Management - What is DSC? - Planning - Performing - Measuring and Evaluating - Supporting Factors.

RE-AIM Framework. RE-AIM: A Framework for Health Promotion Planning, Implementation and Evaluation Are we reaching the intended audience? Is the program.

Developing a Program Logic. What is a Program Logic? A program logic is like a roadmap for a project that sets out how a project will achieve its desired.

IMPLEMENTING LEAPS IN CENTRAL AND EASTERN EUROPE: TRAINERS’ HANDBOOK Monitoring and Evaluating Results.

Using Data to Assess Quality Improvement Results February 23, 2009 Presentation at the CLP/ADRC 2010 Annual Meeting Debra J. Lipson.

1 Outcomes: Outcomes: Libraries Change Lives — Libraries Change Lives — Oh yeah? Prove it. Oh yeah? Prove it. The Institute of Museum and Library Services.

Assessment/Evaluation Make evaluation a central part of planning – not an afterthought 1) Determine Needs 2) Determine Desired Outcomes 3) Determine Activities.

Allison Nichols, Ed.D. Extension Specialist in Evaluation.

What are Learning Outcomes and how to create good Learning Outcomes

Community Gardens Cooking classes Inputs

Designing Effective Evaluation Strategies for Outreach Programs

BEST PRACTICES IN LIBRARY INSTRUCTION FORUM November 7, 2007

Chapter 15 Evaluating Dr. James Pelletier The Swain Department of Nursing.

Utilizing the LOGIC MODEL for Program Design and Evaluation

Short term Medium term Long term

Resources Activity Measures Outcomes

“Methodology SROI & Key Stakeholders”.

Presentation transcript:

A Hierarchy for Targeting Outcomes and Evaluating Their Achievement S. Kay Rockwell Professor and Evaluation Specialist Agricultural Leadership, Education and Communications Department University of Nebraska

Program DevelopmentProgram Performance *SEE Conditions *SEE Outcomes Practices ** KASA ** KASA Reactions Participation Activities Resources Feedback *S = Social *E = Economic *E = Environment* **K = Knowledge A = Attitude S = Skill A = Aspirations Targeting Outcomes of Programs (TOP)

Program DevelopmentProgram Performance *SEE Conditions *SEE Outcomes Practices ** KASA ** KASA Reactions Participation Activities Resources Feedback *S = Social *E = Economic *E = Environment **K = Knowledge A = Attitude S = Skill A = Aspirations Targeting Outcomes of Programs (TOP)

Program Development Program Performance *SEE Conditions *SEE Outcomes Practices ** KASA ** KASA Reactions Participation Activities Resources Feedback *S = Social *E = Economic *E = Environment **K = Knowledge A = Attitude S = Skill A = Aspirations Targeting Outcomes of Programs (TOP) Assess Needs

Program DevelopmentProgram Performance *SEE Conditions *SEE Outcomes Practices ** KASA ** KASA Reactions Participation Activities Resources Feedback *S = Social *E = Economic *E = Environment **K = Knowledge A = Attitude S = Skill A = Aspirations Targeting Outcomes of Programs (TOP)

Stakeholder -a person, inside or outside the organization, who has: a real, active interest in the organization and its programs; an investment in the organization/program (time, mental/emotional energy, money); and a commitment to the organization's/program's success.

Program DevelopmentProgram Performance *SEE Conditions *SEE Outcomes Practices ** KASA ** KASA Reactions Participation Activities Resources Feedback *S = Social *E = Economic *E = Environment* **K = Knowledge A = Attitude S = Skill A = Aspirations Targeting Outcomes of Programs (TOP)

Program vs. Activity Home Study Courses Print Media Multi-media Courses Direct Teaching Video Conferencing Demonstration Sites Publications Tours Programs

Program DevelopmentProgram Performance *SEE Conditions *SEE Outcomes Practices ** KASA ** KASA Reactions Participation Activities Resources Feedback *S = Social *E = Economic *E = Environment **K = Knowledge A = Attitude S = Skill Targeting Outcomes of Programs (TOP)

Inputs Staff and Volunteers Financial Resources Organization/Agency Commitments Grant Dollars

Program DevelopmentProgram Performance *SEE Conditions *SEE Outcomes Practices ** KASA ** KASA Reactions Participation Activities Resources Feedback *S = Social *E = Economic* *E = Environment **K = Knowledge A = Attitude S = Skill A = Aspirations Targeting Outcomes of Programs (TOP) Assess Opportunities

Program DevelopmentProgram Performance *SEE Conditions *SEE Outcomes Practices ** KASA ** KASA Reactions Participation Activities Resources Feedback *S = Social *E = Economic *E = Environment **K = Knowledge A = Attitude S = Skill A = Aspirations Targeting Outcomes of Programs (TOP)

Program DevelopmentProgram Performance *SEE Conditions *SEE Outcomes Practices ** KASA ** KASA Reactions Participation Activities Resources Feedback *S = Social *E = Economic* *E = Environment **K = Knowledge A = Attitude S = Skill A = Aspirations Targeting Outcomes of Programs (TOP)

OUTPUTS = Products or services generated for programming Newsletters Workshops Displays Demonstration Projects Etc. Coalitions developed Curricula developed Consultations Etc.

Program DevelopmentProgram Performance *SEE Conditions *SEE Outcomes Practices *** KASA ** KASA Reactions Participation Activities Resources Feedback *S = Social *E = Economic *E = Environment* **K = Knowledge A = Attitude S = Skill A = Aspirations Targeting Outcomes of Programs (TOP)

Program DevelopmentProgram Performance *SEE Conditions *SEE Outcomes Practices ** KASA ** KASA Reactions Participation Activities Resources Feedback *S = Social *E = Economic *E = Environment* **K = Knowledge A = Attitude S = Skill A = Aspirations Targeting Outcomes of Programs (TOP) Process evaluation

Process Evaluation Aim – provide information for program …improvement …modification …management (Developmental or Formative)

Program DevelopmentProgram Performance *SEE Conditions *SEE Outcomes Practices ** KASA ** KASA Reactions Participation Activities Resources Feedback *S = Social *E = Economic *E = Environment **K = Knowledge A = Attitude S = Skill A = Aspirations Targeting Outcomes of Programs (TOP)

Outcomes = The effects that the products and services (outputs) have on customers or clientele Behavior changes of individuals, groups, organizations, or communities. Attributed in part to the efforts and influences of the agency, program or project (Kibel, 1999)

Program DevelopmentProgram Performance *SEE Conditions *SEE Outcomes Practices ** KASA ** KASA Reactions Participation Activities Resources Feedback *S = Social *E = Economic *E = Environment **K = Knowledge A = Attitude S = Skill A = Aspirations Targeting Outcomes of Programs (TOP) Long-term Intermediate Immediate OUTCOMES

Impacts = Lasting and generalized changes that occur because of the..  Activity(ties)  Program(s)  Grant(s)

Impact Evaluation = Form of outcomes evaluation Assess net effect of a program Compare Program Outcomes Estimate of what would have happened in the absence of the program

Indicators Specific items of information that trace a program’s success They Describe: ….observable characteristics ….measurable characteristics ….or changes that represent achievement of an outcome

Objective Indicators Direct observation of what people overtly do or receive, as well as observations of natural phenomena  Statistical records  Objective tests  Third party observations  Etc.

Subjective Indicators Self-report processes by participants themselves and/or other who may be affected by the program  Example 1: Public satisfaction with personal health; economic status; and cleanness of public air, land, and water  Example 2: reports/ratings by program teams or program participants of their adoption and use of recommended practices and technologies.

Program DevelopmentProgram Performance *SEE Conditions *SEE Outcomes Practices ** KASA ** KASA Reactions Participation Activities Resources Feedback *S = Social *E = Economic* *E = Environment* **K = Knowledge A = Attitude S = Skill A = Aspirations Targeting Outcomes of Programs (TOP)

ProgramP DevelopmentProgram Performance *SEE Conditions *SEE* Outcomes Practices ** KASA ** KASA Reactions Participation Activities Resources Feedback *S = Social *E = Economic *E = Environment **K = Knowledge A = Attitude S = Skill A = Aspirations Targeting Outcomes of Programs (TOP)

Why Assess needs and evaluate? 1.Help establish goals and objectives 2.Test if met goals and objectives 3.Identify outcomes/impact on..  Individuals  Community 4.Provide documentation for stakeholders 5.Document accountability

A Hierarchy for Targeting Outcomes and Evaluating Their Achievement S. Kay Rockwell Professor and Evaluation Specialist Agricultural Leadership, Education and Communications Department University of Nebraska