1. Copyright @ Allyn & Bacon 2003 Social Work Research Methods: Qualitative and Quantitative Approaches Topic 11: Evaluation of Practice Visit the American Evaluation Association: http://www.eval.org/ This multimedia product and its contents are protected under copyright law. The following are prohibited by law: any public performance or display, including transmission of any image over a network; preparation of any derivative work, including the extraction, in whole or in part, of any images; any rental, lease, or lending of the program.

2. Copyright @ Allyn & Bacon 2003 Topic 11: Evaluation of Practice Why Evaluate an Agency? Why Not Evaluate an Agency? Why use Internal Evaluators? Why use External Evaluators? What is the Scope of an Evaluation? What are Summative and Formative Evaluations? What is a Logic Model? What are the basic types of Impact-Outcome Evaluations? What is Single Case Design? What is the basic A-B Design? What is empowerment evaluation? The role of diversity in evaluation. What are the special concerns in evaluation with Non-Euromale Clients? What is a Non-Euromale orientation? What is a EuroMale orientation?

3. Copyright @ Allyn & Bacon 2003 Why Evaluate an Agency? Is the best available evidence being used to support social work interventions? Are resources being well spent? Are services efficient and cost-effective, and are they reaching the target population? How well are client needs being met? Are successes being celebrated? Are clients being empowered? Are ineffective programs being changed or replaced?

4. Copyright @ Allyn & Bacon 2003 Why Not Evaluate an Agency? If goals are poorly defined or unrealistic, or if interventive strategies are not grounded in practice wisdom. If data are unavailable or unusable. If evaluations will be used only to show positive outcomes or for political purposes. If resources are insufficient, if the program is not well developed or if enough time has not passed for an intervention to have taken effect.

5. Copyright @ Allyn & Bacon 2003 Why use Internal Evaluators? Advantages include that internal evaluators are likely to be more familiar, more involved, have institutional memories and are likely positioned to follow up on evaluation recommendations. Disadvantages of internal evaluators include their possible lack of independence and bias; they may know clients, they may have ethical conflicts, they may lack power, and the extra work required may detract from their regular duties.

6. Copyright @ Allyn & Bacon 2003 Why use External Evaluators? Advantages of external evaluators include that they may have superior skills, fresh ideas and new perspectives, in addition to more independence and objectivity. Disadvantages of external evaluators include less knowledge of the organization and limited access to stakeholders; they may also be more expensive and lack the capacity for follow-up.

7. Copyright @ Allyn & Bacon 2003 What is the Scope of an Evaluation? Range - whether an evaluation should involve a centralized or decentralized authority system. Coverage - what kinds of activities, efforts and alternatives should be included or excluded. Time Frame - short-term or long-term assessment. Target population – do those intended to receive an intervention actually receive it. Do others benefit who are not originally targeted (the spillover effect)?

8. Copyright @ Allyn & Bacon 2003 What are Summative and Formative Evaluations? Summative evaluation looks at existing information across a number of areas to see how well an existent system has functioned via retrospective assessment. Formative evaluation looks at what can be done differently in the future by a prospective assessment.

9. Copyright @ Allyn & Bacon 2003 What is a Logic Model? The Logic model is a method for graphically presenting links between program goals, efforts and expected outcomes. It graphically illustrates an agencys Inputs - Activities-Outputs-Initial and Longer-Term Outcomes.

10. Copyright @ Allyn & Bacon 2003 What are the basic Types of Impact/Outcome Designs? Experimental - a classical or other major design involving treatment and control groups. Quasi-experimental - a variation of classical design missing one or more elements. Non-experimental - involves before/after comparisons, time series, and cross sectional comparisons.

11. Copyright @ Allyn & Bacon 2003 Other Evaluation Designs Monitoring Performance - gives feedback on how a program or series of interventions is operating, particularly with regard to quality issues. Process Evaluation - assesses how a program operates, how services are delivered, and how well the interventive model is being delivered. A Case Study - a detailed analysis of one or a few selected program sites or clients who are representative of those not selected.

12. Copyright @ Allyn & Bacon 2003 What is Single Case Design? A combination of research and practice that assesses the progress of a single client or case observed during both interventive and non-interventive periods.

13. Copyright @ Allyn & Bacon 2003 What is The Basic A-B Design? A Period is a time of clinical baseline observation and measurement of the clients condition or targeted behaviors. No intervention takes place during this period; it is analogous to the control group. B Period is a time of clinical of intervention, observation and measurement in which the treatment is introduced; it is analogous to an experimental group.

14. Copyright @ Allyn & Bacon 2003 What is Empowerment Evaluation? Empowerment research blends research and evaluation from a critical theory and feminist perspective with social and economic justice. There is no such thing as completely objective, disinterested assessment in empowerment research.

15. Copyright @ Allyn & Bacon 2003 What are the special concerns Evaluating Non-Euro-Centered Clients? Stress the historical understanding of oppression suffered. Use the ecological perspective, especially adaptive functioning capacities. Obtain knowledge of unequal power arrangements and how oppression is often internalized. Focus on effects of socioeconomic class, race and gender. Attend to the link between individual pain and strategies for change.

16. Copyright @ Allyn & Bacon 2003 What is a Non-Euro-Male Orientation? The non-Euro-Male Orientation focuses on group or tribal references, the larger community, naturalism, mother earth and father sky, a process rather than product orientation, cooperation, the extended family, benevolence and sharing, polychronic time, oral traditions, spoken as opposed to written language and giving voice to those traditionally unheard (the bottom-up perspective).

17. Copyright @ Allyn & Bacon 2003 What is a Euro-Male Orientation? The Euro-Male Orientation focuses on individualism, autonomy of the self, empiricism, masculine spirituality, product orientation, competition, performance teams, achievements and affluence, monochronic time, formal codes, the written word, and authority and control discussed from a top down perspective.