Program Evaluation Dr. Ruth Buzi Mrs. Nettie Johnson Baylor College of Medicine Teen Health Clinic.

Slides:



Advertisements
Similar presentations
Muskie School of Public ServiceInstitute for Public Sector Innovation Where we are trying to go? Common outcome measures Quality data collection for outcomes.
Advertisements

Sexual health education David Ross London School of Hygiene & Tropical Medicine Bergen, 7 th May 2014.
Comprehensive Organizational Health AssessmentMay 2012Butler Institute for Families Comprehensive Organizational Health Assessment Presented by: Robin.
Teenage Pregnancy… An educator's role in prevention
Focus on Kids Intervention for Adolescents in High School to Prevent STDs/HIV Charlotte A. Gaydos, Dr.P.H. Johns Hopkins University Baltimore, Maryland.
Integration: Intersection for Reproductive Health and HIV Programs: the Kenyan Experience Family Health International Sponsored Satellite Session World.
Polokwane, 28 September WORKING WITH ADOLESCENTS WHO defines adolescence as years Divided into sub-periods (early, middle and late) Rate of.
Healthy Living Gr. 8. Healthy Living Outcomes  8.HLIV8.O.1.1- analyze the relationship between values and personal health practices  8.HLIV8.O.2.10-
Developing a Logic Model
Dennis McBride, Ph.D. The Washington Institute (253) Goal Driven Logic Models.
Linda Chamberlain, PhD MPH IPV and Sexually Transmitted Infections/HIV MENU Overview Regional and Local Data The Impact of IPV on Women’s Health IPV and.
HIV Counseling. Required at all study visits  Pre-test counseling  Post-test counseling  Risk reduction counseling  Condom Provision Will you incorporate.
Community Planning Training 1-1. Community Plan Implementation Training 1- Community Planning Training 1-3.
Implementing Rapid HIV Testing in New York State Mara San Antonio-Gaddy Director Bureau of Direct Program Operations NYSDOH, AIDS Institute.
Midwest AIDS Training & Education Center Health Care Education & Training, Inc. HIV/AIDS Case-Finding In Family Planning Clinics.
The Body of Power Dr. Kimberly Brodie Health Behavior Theory MPH 515 Sara Quale Dec. 16, 2013 Combatting the Spread of Sexually Transmitted Disease in.
How to Develop the Right Research Questions for Program Evaluation
Monitoring the effects of ARV treatment programmes on prevention Gabriel Mwaluko 1, Mark Urassa,2, John Changalucha,2, Ties Boerma 3 1 TANESA Project,
Susan Adamchak, Heidi Reynolds, Barbara Janowitz, Thomas Grey, Emily Keyes October 21, 2008 FP and HIV/AIDS Integration: Findings from 5 Countries.
Activities Outcomes – Impact Short Term Medium Term Long Term Problem statement: New York City teen pregnancy rates are higher than the national average,
Case Management 1 What Will be Covered? 1. Organizational Assessments (Module 1) 2. Designing Quality Services (Module 2) 3. HIV Prevention and Care Advancing.
Sociology 3322a. “…the systematic assessment of the operation and/or outcomes of a program or policy, compared to a set of explicit or implicit standards.
Development and results of an older adult health communication program using the Theory of Planned Behavior Virginia Brown, DrPH; Lisa McCoy, MS The National.
Program Evaluation and Logic Models
Fundamentals of Evaluation for Public Health Programs ROBERT FOLEY, M.ED. NIHB TRIBAL PUBLIC HEALTH SUMMIT MARCH 31,
Analyzing the Teen Wellness Program using the YRBS Prevention Research Center for Healthy Neighborhoods, Case Western Reserve University Gloria Agosto.
Ensuring the Fundamentals of Care in Family Planning and Reproductive Health Services MODULE 2 Facilitative Supervision for Quality Improvement Curriculum.
HIV/AIDS BI-ANNUAL REVIEW 2008 Prevention -Goal, Indicators and Targets TACAIDS.
DHSTS April 28, 2014 Annual HIV Testing Project Coordinator’s Meeting.
Office of Overseas Programming & Training Support (OPATS) Positive Health, Dignity, and Prevention HIV Prevention.
Effect of a values-based prevention curriculum on HIV- positive couples from four regions in Ethiopia Presented at XIX IAC 2012 By Misgina Suba, MPH 25.
Supporting HIV positive mothers with infant feeding issues Group 4.
Youth Mental Health and Addiction Needs: One Community’s Answer Terry Johnson, MSW Senior Director of Services Senior Director of Services Deborah Ellison,
HIV Prevention Programs That Work Centers for Disease Control and Prevention (CDC)
Monitoring &Evaluation System in Health Program. Brief overview of NACP Reporting units and quality aspects Data sets Learning/ Analysis of the data sets.
HIV/AIDS Education Impact on Risky Adolescent Sexual Behaviors across Racial Groups Seventh National Conference on Quality Health Care for Culturally Diverse.
Nancy L. Weaver, PhD, MPH Department of Community Health School of Public Health Saint Louis University 16 July 2010 LOGIC MODEL FUNDAMENTALS.
Reasoned Abstinence Plus Focus group: y/o Female Hispanic and African American RAP will be presented for implementation to the SHAC of zip codes:
Prevention Interventions for Substance Abuse Professionals.
Methods for the evaluation of the Abstinence and Faithfulness among Youth (ABY) interventions in South Africa Dr Geoff Setswe, Dr Mark Ottenweller & Roger.
Child/Youth Care Management 2015 training. WELCOME!
Brianna Loeck Principles of Health Behavior - MPH 515 Kimberly Brodie August 22, 2013 Educate Prevent Sexually Transmitted Diseases & Teen Pregnancy.
21/4/2008 Evaluation of control measures 1. 21/4/2008 Evaluation of control measures 2 Family and Community Medicine Department.
Family Resource and Youth Services Centers: Action Component Plan.
Lessons from the CDC/RTC HIV Integration Project Marianne Zotti, DrPH, MS, FAAN Team Leader Services Management, Research & Translation Team NCCDPHP/DRH/ASB.
WAIT Training Annual Report Educational Evaluators Inc. Expertise.Partnership.Results.
Towards No Drug Use (TND) Program Description  Project TND: A series of 12 lessons implemented in all high schools  25 teachers from 14 high schools.
Project KEEP: San Diego 1. Evidenced Based Practice  Best Research Evidence  Best Clinical Experience  Consistent with Family/Client Values  “The.
HIV Counseling Tab 7 of the binder..
Using Logic Models to Create Effective Programs
Prevention Education Meeting May 29, 2013 Evaluation 101.
Efficacy of a “One-Shot” Computerized, Individualized Intervention to Increase Condom Use and Decrease STDs among Clinic Patients with Main Partners Diane.
Families Matter! Program Increases Parent-Preadolescent Communication about Sex in Tanzania An Outcome Evaluation in Mtwara and Ruvuma Shoo T, Kamala B,
Marlene Luna Mentor: Britt Rios-Ellis, PhD Department of Health Science California State University, Long Beach HIV / AIDS Latinas.
HIV/AIDS: Risk perception and Sexual Practices of Barbadian teenagers.
Session 5: Selecting and Operationalizing Indicators.
Crystal Reinhart, PhD & Beth Welbes, MSPH Center for Prevention Research and Development, University of Illinois at Urbana-Champaign Social Norms Theory.
Andrea Moore Information Specialist MANILA Consulting Group, Inc. American Evaluation Association Annual Meeting November 11, 2010 The Community-based.
Ethiopia Demographic and Health Survey 2011 HIV/AIDS Knowledge, Attitudes, and Behaviour.
So You Think You’ve Made a Change? Developing Indicators and Selecting Measurement Tools Chad Higgins, Ph.D. Allison Nichols, Ed.D.
SRHR Policy Salima 30 th June 2011 SRHR Policy Salima 30 th June 2011 Foundation for Children Rights.
Peer Education Plus (PEP) Model; A Veritable Tool for achieving Behaviour Change. Experience from a Rural Community In Kaduna State, Nigeria AUTHORS: P.
Non-Medical Staff Knowledge, Beliefs and Practices about HIV and Hepatitis for Injection Drug Users Rowe, KA 1, Tesoriero, JM 1, Heavner, KK 1, Rothman,
Provider Initiated HIV Counseling and Testing Unit 2: Introduction and Rational for PIHCT.
Promoting Science-based Approaches to Preventing Teen Pregnancy, STDs and HIV Policy, Partnerships, and Creativity Brigid Riley, MPH American Public Health.
1 Module 2: HIV Counseling and Testing for PMTCT Ministry of Health/HAPCO, Ethiopia.
OUTCOME MONITORING A Short Review Nidal Karim. What is Outcome Monitoring? WhatHowWhyWhen The routine process of monitoring whether or not service recipients.
Sherry Deren, Sung-Yeon Kang, Milton Mino & Honoria Guarino
Presentation transcript:

Program Evaluation Dr. Ruth Buzi Mrs. Nettie Johnson Baylor College of Medicine Teen Health Clinic

Program Goals Encourage adolescents to engage in prevention interventions to adopt and maintain HIV risk reduction behaviors by selecting: Encourage adolescents to engage in prevention interventions to adopt and maintain HIV risk reduction behaviors by selecting: A -Abstinence A -Abstinence B - Be Faithful B - Be Faithful C onsistent & Correct Condom use C onsistent & Correct Condom use

Logic Model Using a logical framework model to evaluate program outcomes. This includes: Using a logical framework model to evaluate program outcomes. This includes: Program Inputs Program Inputs Program Activities Program Activities Program Outputs Program Outputs Initial Outcomes Initial Outcomes Intermediate Outcomes Intermediate Outcomes Long-term Outcomes Long-term Outcomes

Program Inputs Agency provides Risk Reduction Specialists at clinic sites Agency provides Risk Reduction Specialists at clinic sites

Program Activities Program provides client-centered Individual Level Intervention focused on assisting adolescents to select an appropriate risk reduction plan. Program provides client-centered Individual Level Intervention focused on assisting adolescents to select an appropriate risk reduction plan.

Initial Outcomes Initial Outcomes Adolescents are knowledgeable about HIV risk and personal risk and develop a risk reduction plan. Adolescents are knowledgeable about HIV risk and personal risk and develop a risk reduction plan. Adolescents acquire more favorable attitudes about abstinence and condoms. Adolescents acquire more favorable attitudes about abstinence and condoms. Adolescents demonstrate an intent to use condoms. Adolescents demonstrate an intent to use condoms. At-risk adolescents know their HIV status. At-risk adolescents know their HIV status.

Intermediate Outcomes Intermediate Outcomes Adolescents follow a risk reduction plan as reflected by an increase in abstinence, consistent condom use, and a reduction in the number of sexual partners. Adolescents follow a risk reduction plan as reflected by an increase in abstinence, consistent condom use, and a reduction in the number of sexual partners.

Long Term Outcomes Long Term Outcomes A reduction in sexually transmitted infections (STIs) and HIV rates among adolescents attending the Teen Health Clinic. A reduction in sexually transmitted infections (STIs) and HIV rates among adolescents attending the Teen Health Clinic.

Quantitative Measures Knowledge and Attitude- An ABC pre- post 15-item self-administered instrument is a rapid measurement technique that examines the following: Knowledge and Attitude- An ABC pre- post 15-item self-administered instrument is a rapid measurement technique that examines the following: Attitudes about sex during adolescence Attitudes about sex during adolescence Knowledge about HIV transmission Knowledge about HIV transmission Intention to use condoms in the future Intention to use condoms in the future

Examples of Knowledge and Attitude Changes QuestionPrePost Condoms can prevent HIV transmission 85.2% True 92.7% True Abstinence is the only 100% way to prevent HIV 80.0%87.3% The risk of HIV is a reason to avoid sex 45.5% SA 50.9% SA Sex after long dating is OK 34.5% SD 47.3% SD Intention to use condoms 80.0%83.6%

Quantitative Measures Behavior Change: The Youth Risk Behavior Survey (YRBS) measures the following: Behavior Change: The Youth Risk Behavior Survey (YRBS) measures the following: Number of sexual partners Number of sexual partners Condom use Condom use History of STIs History of STIs Sexual activity (abstinence) Sexual activity (abstinence) Attitudes about condoms Attitudes about condoms

Quantitative Measures YRBS collects baseline data on pre-existing trends and examines changes over time (year one, year two, etc.). YRBS collects baseline data on pre-existing trends and examines changes over time (year one, year two, etc.). YRBS is administered at baseline and follow-up (three and six months) to examine direct program impact. YRBS is administered at baseline and follow-up (three and six months) to examine direct program impact.

Examples of trend surveys Indicator Year 1 Year 2 Recent condom use 45%41.7% Lifetime sexual partners Sexual partners (3 months) Intention to use condoms 59.1%70% History of STIs 44%31.6%

Examples of Program Impact IndicatorBaselineFollow-up Recent condom use 45%47.4% Lifetime sexual partners 4.6N/A Sexual partners (3 months) Intention to use condoms 59.1%75.9%

Program Monitoring How many clients (unduplicated) are receiving ABC prevention counseling? How many clients (unduplicated) are receiving ABC prevention counseling? Are clients receiving the proper amount, type, and quality of services? Are clients receiving the proper amount, type, and quality of services? Do participants engage in appropriate ABC follow-up sessions? Do participants engage in appropriate ABC follow-up sessions? Are participants satisfied with program procedures? Are participants satisfied with program procedures?

Program Monitoring-Qualitative Assessment Qualitative methods can be useful for obtaining knowledge on the specific needs of a population. Qualitative methods can be useful for obtaining knowledge on the specific needs of a population. Qualitative data on the structure of beliefs or attitudes of the target population can contribute substantially to the effective design of educational campaigns. Qualitative data on the structure of beliefs or attitudes of the target population can contribute substantially to the effective design of educational campaigns.

Program Monitoring-Qualitative Assessment Focus groups consisting of adolescents who received ABC education were conducted to evaluate the relevance of the ABC approach with adolescents. Focus groups consisting of adolescents who received ABC education were conducted to evaluate the relevance of the ABC approach with adolescents. The purpose was to obtain their reaction to the ABC message and to the delivery of that message in a teen family planning clinic setting. The purpose was to obtain their reaction to the ABC message and to the delivery of that message in a teen family planning clinic setting.

Focus Groups Results suggested that overall, teens were positive about continuing to use the ABC message that offers them choices and options for reducing their risk, rather than directives. Results suggested that overall, teens were positive about continuing to use the ABC message that offers them choices and options for reducing their risk, rather than directives. Although teens agreed that abstinence is a good choice, they felt it can be challenging at times. Although teens agreed that abstinence is a good choice, they felt it can be challenging at times.

Focus Groups Teens suggested that peer pressure is an important factor in teen decision making about sexual activity. Teens suggested that peer pressure is an important factor in teen decision making about sexual activity. Teens felt that parents’ role is important and that parents should start discussing issues related to sex with their children at an early age. Teens felt that parents’ role is important and that parents should start discussing issues related to sex with their children at an early age.

Evaluation Challenges Social desirability bias on pre-post tests Social desirability bias on pre-post tests Low compliance with follow-up care interferes with collection of post-test data Low compliance with follow-up care interferes with collection of post-test data A clinic-based study versus a population- based study is limited in documenting abstinence as adolescents who choose abstinence following intervention will not return to the clinic. A clinic-based study versus a population- based study is limited in documenting abstinence as adolescents who choose abstinence following intervention will not return to the clinic. Evaluation is costly and time-consuming. Evaluation is costly and time-consuming.

Actions to Improve Evaluation Developing better standardized instruments to measure the effects of the ABC intervention. Developing better standardized instruments to measure the effects of the ABC intervention. Develop better indicators to measure program impact. Develop better indicators to measure program impact. Strengthening data collection systems and standardized measures to reduce the costs of data collection. Strengthening data collection systems and standardized measures to reduce the costs of data collection.