Research in Social Work Practice Salem State University Class 3 Jeff Driskell, MSW, PhD

Today’s Class Check-in/announcements Lecture/activities Evaluation

Program Development and Evaluation

Program Lifecycle

Breaking it Down… Program Development- An ongoing, comprehensive planning process used to establish programs supported by well thought out and documented plan of action Program Implementation- The stage in the planning process where you develop procedures for carrying out program. Program Evaluation- Designed to verify, document, quantify activities and efforts. Process and outcome evaluation. Replication- Replicating evidenced-based programs.

Development Needs assessment Clearly defined audience/target population Identify behavioral theory Identify behavioral domains Identify behavioral/program goals and objectives

Culturally Competent Program Development Assessing needs and assets Defining the problem Clearly defined audience/target population Identification/Application of behavioral domains (theoretical concepts) Identifying goals and objectives Planning program components Using a program model/framework Staffing/training Setting

Implementation Select and implement behavioral interventions Variety of teaching methods Information personalized Realistic schedule Repetition of the message Evaluation

Example #1- Project Enhance Overall Goal- To reduce rates of HIV infection by 25% among HIV negative and unknown status MSM by 2010 in Boston, MA. Randomized control trial targeting HIV positive MSM involved in health care (setting) Secondary prevention (level of prevention) IMB Model (primary theoretical framework) Facilitated by Medical Social Worker (staffing) Utilizes a manual treatment approach (delivery method) Consists of 8 counseling sessions- over a 1 year period of time (Dose) Change in sexual risk behavior (outcome evaluation)

Project Enhance Booster, PCM & Assessment PCM & Assessment Month 3: Month 6: Month 9: Month 12: randomized to PCM randomized to intervention Booster, PCM & Assessment PCM & Assessment PCM follow-up & Intervention Session 1 Consent, Assessment Randomization & Baseline PCM PCM follow-up PCM follow-up & Intervention Sessions 2,3 & 4

Culturally Competent Program Implementation Involvement of community leaders, consumers Staff recruitment and training Responding to target population needs Addressing potential language barriers Retaining program staff Participant recruitment Monitoring of program

Evaluation

Why Conduct Evaluation?

Types of Evaluation Evaluation of Need Evaluation of Process Evaluation of Efficiency- cost/benefit Evaluation of Outcome

Evaluation Categories: Matching Activity Evaluation of Need Evaluation of Process Evaluation of Efficiency- cost/benefit Evaluation of Outcome Formative Summative

Mapping it Out…

Process Evaluation: The Milestone Program

Milestone Program Target Population- HIV infected men Service- Prevention Case Management Evaluation type- Process Community Advisory Board (CAB)- consumer voice and input Internal vs external evaluators- Conducted by University of Washington

Experience with Evaluation Evaluation at past or currently place of employment/field.

Activity 1: Weekly Comments Based on the Olver (2009) article, answer the following questions: What problem is being addressed? What type of evaluation? Describe the intervention Identify data collection strategies What are the implications for practice? Policy? Research?

Practice Relevance: Application Scenario: A male client in his mid 30’s approaches you as his therapist and informs you that he is interested in participating in a workshop called the Mankind Project. He is a bit ambivalent and asks what you know about it. What might you do in supporting this client? Think in terms of Evidenced based practice and evaluation. Mankind Project

Logic Models

What is a Logic Model? Based on the reading for this week… Rubin and Babbie

Logical Model Overview http://www.youtube.com/watch?v=OZ2oBVCDU9I

Framework for Evaluation Systems approach Inputs- What you invest Outputs- What you do and who you reach Activities Participation Outcomes/impact

Simplest Form INPUTS OUTPUTS OUTCOMES Participation What we invest Activities Participation Short Medium Program investments Long-term What we invest What we do Who we reach What are the results

Logical Model Example https://www.youtube.com/watch?v=KEV2Km23MFk

Fully Detailed Logic Model Typical layout

Activity - Logical Model http://www.uwex.edu/ces/pdande/evaluation/evallogicmodelexamples.html Break it down- What elements are associated with each stage of the logic model example? Select the tobacco prevention for youth

Activity- Evaluation and EBP http://nrepp.samhsa.gov/AdvancedSearch.aspx Step 1 What is NREPP? What does NREPP provide in relation to interventions? Step 2 Identify 1 program of interest using advanced search Briefly describe the program Identify 2 key findings based on targeted outcomes (key findings)

Activity – Student Articles Identify the type of research, i.e. Outcome evaluation Process evaluation Needs assessment Questions- What question(s) were the researchers trying to answer with the evaluative research? What are the study or program goals/objectives? Where the goals/objectives achieved? What are the implications for practice? Policy?

Application: Logic Model

Weekly Comments Describe what is mean by a literature review. What is the purpose of a literature review? What strategies do Rudestam & Newton (2001) give for organizing and conducting a literature review? What is meant by the concept mixed methods design?