Evaluation and Assessment C H A P T E R 4 Evaluation and Assessment Chapter 4

Outcomes Understand the eight steps of evaluation. Understand the purpose of needs assessments and how they can be utilized in a leisure services agency. Describe different methods of data collection for evaluations and needs assessments. (continued)

Outcomes (continued) Define qualitative and quantitative data and provide examples of each. Describe different types of evaluation models that can be used for recreation programs and facilities. Understand the process for developing an evaluation report, including the types of information that should be included.

Eight Steps of the Research Process Identify the problem Review the literature Clarify the problem Define terms and concepts (continued)

Eight Steps of the Research Process (continued) Define the population Develop an instrumentation plan Collect the data Analyze the data

Important Evaluation Terms formative evaluation—An evaluation that provides feedback on an ongoing program. summative evaluation—An evaluation that provides final program feedback. reliability—The quality describing instrument and data consistency in measurements. validity—The quality describing data that provide true measurement of reality.

Models of Evaluation Importance–performance evaluation: compares agency performance on variables of interest to importance of variables as rated by participants (continued)

Models of Evaluation (continued)

Models of Evaluation (continued) Satisfaction-based evaluation: provides measures related to the different potential outcomes of leisure engagement Service hour evaluation: primarily concerned with outputs, such as how many people the agency is serving and to what extent Goals and objectives (process) evaluation: compares the plan of a program to the actual operation or outcomes of the program

Purposes of Needs Assessment Determine satisfaction with parks, programs, facilities Measure interest in future parks, programs, facilities Determine marketing effectiveness and needs Determine willingness to pay

Types of Data Qualitative: nonnumerical data such as interview data Quantitative: numerical data

Data Collection Methods Citizen advisory committees Public meetings and focus groups Interviews and key informants Observation Existing records (e.g., registration forms)

Data Collection Methods: Nominal Group Technique Participant selection Individual brainstorming Round-robin listing Individual ranking of top five priorities Rankings tabulation Discussion of ideas in context of rankings

Data Collection Methods: Surveys Paper versus Web-based surveys Delphi technique Select experts. Create and distribute first questionnaire. Create and distribute second questionnaire. Create and distribute third questionnaire. Reach consensus.

Data Analysis Quantitative: statistical analysis using computer software such as Excel Qualitative: examination of data for themes or trends

Evaluation Report Know your audience and its expectations Purpose statement to provide context Program description and applicable regulations Background of evaluators (continued)

Evaluation Report (continued) Methods used in data collection Findings from the evaluation Relevant appendices

Outcome-Based Programming Program goals and objectives are tied to anticipated program outputs and outcomes and ideally tied to program theory. Outputs are the direct products of the program. Outcomes are the effects (benefits and changes) that the program has on participants.

Logic Model for Outcome-Based Programming

Outcome-Based Assessment May be formative as well as summative Tied to theory driving the programming Measurement of both outputs and outcomes as a result of participation in the program