Evaluation of Community Interventions Running programs and finding out if they work

Quantitative & Qualitative approaches include: Experimental Designs Quasi-Experimental Designs Pre & Post test studies Time Series Analysis Social Indicator Analysis Longitudinal Study Survey Satisfaction Survey Goal Attainment Program Monitoring Ethnographic Study Feminist Research Constructivist Evaluation Process Analysis Implementation Analysis Focus Groups Community Forums Critical incidence analysis Key informant interviews Field interviews Personal narratives

Use the following criteria to determine type of evaluation Research question to be addressed. Amount of resources and time that can be allocated for research. Ethics (can you reasonably construct control groups or hold confounding variables constant) Will the evaluation be conducted by an internal or external evaluator? Who is audience for the evaluation? How will the data be used? Who will be involved in the evaluation?

Types of evaluation approaches that involve organization constituents Participatory Action Research Empowerment Evaluation Self-evaluation

Differences in approaches are: Participatory Action Research Empowerment Evaluation Self-Evaluation Role of Researcher Consultant; Partner with participants Consultant; works for participants Consultant; works for agency/funder PurposeSocial ChangeSelf- Determination Evaluate Agency Services OutcomeAlleviate Oppression Increases Participant Skills and Control Improved Service Quality

Advantages of Methods Increases feelings of participant ownership of process/programs. Increases likelihood that data will be used. Increases likelihood that the resulting program or intervention will meet needs of stakeholders and be culturally appropriate. Participants develop skills and confidence. They gain knowledge and information and thus become empowered.

Disadvantages of Method Distrust and conflict among participants. Length of time needed to develop consensus around goals, mission, and methods. The need for training around research methods, data collection, and analysis. The need for skilled facilitation, coordination, and follow-up on task completion. Money and an organizational structure are needed to do all these things. The group must be able to apply findings in order to achieve an outcome

To illustrate goal development and evaluation, let’s use one of the following models to specify goals, objectives, and evaluation criteria:

Model Community Development Social ActionSocial PlanningTransformative Model Primary Strategy CollaborationContest/ Campaign Problem-solvingPopular education TacticsCapacity building Consensus Education Persuasion Empowerment Bargaining Negotiation Demonstrations Mild Coercion Mobilizing Lobbying Politics Fact gathering Data analysis Citizen participation Negotiation Developing a critical consciousness Transforming oneself Taking action to transform society Action System Residents Group Representatives Constituents Participants Constituents Govt. agencies Partner groups Members of oppressed groups Target System Same as target system Power elite Decision makers Communities Decision makers Oppressors

Many of the approaches used to evaluate community interventions are the same as those used to evaluate programs. Goal attainment. Social Indicator Analysis Case Studies. Process evaluations/implementation analysis. Outcome evaluation (pre-post tests; experimental/control groups, etc.). Consumer satisfaction surveys.

Methods specific to community organization are: Field Interviews Critical Incidence Analysis Social indicator analysis that compares neighborhoods to one another. Historical case studies