Evaluating Your Home Visiting Program

Slides:



Advertisements
Similar presentations
Evaluation Capacity Building Identifying and Addressing the Fields Needs.
Advertisements

+ District of Columbia Department of Health Home Visitation Program.
Affordable Care Act Tribal Maternal, Infant, and Early Childhood Home Visiting Program Overview Moushumi Beltangady Administration for Children and Families.
Affordable Care Act Maternal, Infant, and Early Childhood Home Visiting Program Audrey M. Yowell, Ph.D., M.S.S.S. Chief; Policy, Program Planning and Coordination.
Linking Actions for Unmet Needs in Children’s Health
“If you don’t know where you are going, how are you gonna’ know when you get there?” Yogi Berra Common Elements of Strong Program Evaluation THEORY OF.
Money: NPCC Community Grants Program Rosalina James, PhD.
Oral Health Project Activities to Support Providers and National Partnerships Kathy Geurink, R.D.H., M.A.
WRITING THE SUCCESSFUL PROPOSAL C. June Strickland, Ph.D., RN Associate Professor University of Washington School of Nursing.
Maternal, Infant, and Early Childhood Home Visiting Program Expansion Grant FOA HRSA July 12, 2013 U.S. Department of Health and Human Services.
Affordable Care Act Maternal, Infant, and Early Childhood Home Visiting Program Health Resources and Services Administration Administration for Children.
Overview of Evaluation Designs. Learning objectives By the end of this presentation, you will be able to: Explain evaluation design Describe the differences.
Joan O’Connell, PhD Circles of Care Evaluation Technical Assistance Center American Indian and Alaska Native Programs University of Colorado at Denver.
State and Regional Approaches to Improving Access to Services for Children and Youths with Epilepsy Technical Assistance Conference Call Sadie Silcott,
0 Personnel Development to Improve Services and Results for Children with Disabilities PERFORMANCE MEASURES Craig Stanton Office of Planning, Evaluation,
Math and Science Partnership Program Approaches to State Longitudinal Evaluation March 21, 2011 San Francisco MSP Regional Meeting Patty O’Driscoll Public.
Office of Special Education Programs U.S. Department of Education GRANT PERFORMANCE REPORT FOR CONTINUATION FUNDING.
The HMO Research Network (HMORN) is a well established alliance of 18 research departments in the United States and Israel. Since 1994, the HMORN has conducted.
Evaluation of the Indiana ECCS Initiative. State Context Previous Early Childhood System Initiatives –Step Ahead –Building Bright Beginnings SPRANS Grant.
Grant Application Process Maternal, Infant & Early Childhood Home Visiting Programs.
Good Start, Grow Smart Inter-American Symposium Understanding the State of the Art in Early Childhood Education and Care: The First Three Years of Life.
Become an ACF Grant Reviewer Here are descriptions of key ACF programs and web links for reviewer registration. Administration on Children, Youth and Families.
National Coordinating Center for the Regional Genetic Service Collaboratives ( HRSA – ) Joan A. Scott, MS CGC, Chief, Genetics Services Branch Division.
SUSTAINABLE & HEALTHY COMMUNITIES RESEARCH PROGRAM Highlights Six awards were made in summer Awardees are completing their first year of work, primarily.
Evidence-Based Public Health in Action: Strategies from New York Moderator: Amy Ramsay Association of State and Territorial Health Officials Speakers from.
[Presentation location] [Presentation date] (Confirm ABT logo) Building Bridges and Bonds (B3): An introduction.
Increased # of AI/AN receiving in- home environmental assessment and trigger reduction education and asthma self-management education Increased # of tribal.
Jeopardy Game - Sample This is an example of a jeopardy game that could be used during data collection training. This is an example of a jeopardy game.
Welcome to Workforce 3 One U.S. Department of Labor Employment and Training Administration Webinar Date: April 30, 2014 Presented by: U.S. Departments.
Welcome! These slides are designed to help you think through presenting your evaluation planning and results. Feel free to pick and choose the slides that.
Methodological Issues in Needs Assessment for Quality Assurance in a National Context: The Case of Head Start Needs Assessment Hsin-Ling (Sonya) Hung,
Dennis Embry, PhD Paxis Institute Kate Lyon, MA James Bell Associates, Inc. Aleta Meyer, PhD Office of Planning, Research, and Evaluation Administration.
Welcome! These slides are designed to help you think through presenting your benchmark planning and progress. Feel free to pick and choose the slides that.
Building Capacity to Conduct Scientifically and Culturally Rigorous Evaluations in Tribal Communities through the Tribal Home Visiting Evaluation Institute.
Kent CHAP History Health Net of West Michigan. Kent CHAP History Health Net of West Michigan.
Administration for Children and Families
The Kansas Kindergarten Readiness Initiative: The Special Educator’s Role Barb Dayal Vera Stroup-Rentier.
Sharing your CQI Story: Creating a CQI Story Board Tribal MIECHV Annual Grantee Meeting Washington, DC May 6, 2015.
Pathways for Advancing Careers and Education (PACE): Findings from a Study of a Career Pathways Program Karen Gardiner Abt Associates, Inc. National Association.
Building Processes for Conducting and Managing Data Collection
Montessori Research Initiative
Strategies for Supporting Home Visitors with Data Collection
American Evaluation Association Annual Conference
DCF Initiatives to Prevent and Intervene in Youth Homelessness
Putting Your Data to Work
Using mixed methods to develop and evaluate public health education interventions Presented by: Louise C. Palmer
Supporting Community Priorities and Emphasizing Rigor An Approach to Evaluation Capacity Building with Tribal Home Visiting Programs Kate Lyon, MA Julie.
Annual Title I Parent Orientation Meeting
Tribal Home Visiting Evaluation Institute
The Kansas Kindergarten Readiness Initiative: The Special Educator’s Role Barb Dayal Vera Stroup-Rentier.
WIC Dental Days A collaborative Early Childhood Caries prevention program Presented by Theresa Anselmo, Linda McClure, and Suzanne Russell San Luis Obispo.
Kate Lyon, MA, James Bell Associates, Inc.
Building Tribal Capacity for Home Visiting Evaluation through a Relational Technical Assistance Approach American Evaluation Association Annual Conference.
Race to the Top—Early Learning Challenge Letters of Support Webinar
Measuring Success Toolkit
Family-Guided Routines-Based Intervention Introduction Module
Improving Data, Improving Outcomes 2016
Centers for Disease Control (CDC) Tribal Advisory Committee Update and Public Health Initiatives Amy Groom, MPH National Center for Chronic Disease Prevention.
Crosswalk of the Recommendations from the Board of Regents Early Childhood Workgroup’s Blue Ribbon Committee & First 1,000 Days on Medicaid Healthy Children:
Income Eligible Re-Procurement
2016 AmeriCorps Texas All-Grantee Meeting February 25-26, 2016
Income Eligible Re-Procurement
2018 OSEP Project Directors’ Conference
Bureau of Family Health: Infant Toddler Services
State Driven Research to Answer Key Policy Questions
Title Plans and Assurances Spoke Committee Presentation
IV-E Prevention Family First Implementation & Policy Work Group
Tell A Meaningful Story With Data Through Research
Division of Behavioral Health
Linda Mayo Willis and Carolyn Pope Edwards
Presentation transcript:

Evaluating Your Home Visiting Program Kate Lyon, MA James Bell Associates Tribal Home Visiting Evaluation Institute Cohort 3 Kickoff Meeting December 2013

program thriving children community family strong communities Our objectives: thriving children community program healthy families strong communities family

Federal Guidance The goals of evaluating home visiting in your communities are to learn how well it works in YOUR community and to add to body of evidence about effective interventions in AIAN communities Involve community in the evaluation Must conduct rigorous evaluations Flexibility to focus on targeted elements related to unique goals Build knowledge for implementing evidence based HV services in AIAN communities

homvee.acf.hhs.gov HomVEE Which models show effect in each outcome domain? Summaries across models. Information on the studies included in review.

Effects Shown in Research Which models show effect in reducing child maltreatment?

Study Search What studies have been done on Parents as Teachers? 29 moderate or high quality studies reviewed. Complete citations available.

Tribal HomVEE Report Identified studies of home visiting programs implemented in tribal communities for review Of these, none met the criteria for evidence of effectiveness RCTs experienced too much sample attrition; QEDs could not determine baseline equivalence Valuable information about implementation lessons learned http://homvee.acf.hhs.gov/Tribal_Report_2012.pdf

Contributing to the Knowledge Base Evaluation allows you to determine the impact of the intervention in your community Select an evaluation design that allows you to establish causality Focus on outcomes or implementation supports and strategies

Increased adherence to prenatal care Improved birth outcomes opulation I ntervention C omparison O Increased adherence to prenatal care utcomes Improved birth outcomes

PICO Example Do urban American Indian children ages 2-5, living below the poverty level (P) whose families receive Parents as Teachers home visitation services (I) demonstrate greater school readiness (O) compared to children whose families receive usual services (C)? The intervention is developed/selected because it links the target population’s needs to the desired outcomes through a theory of change.

PICO PICO exercise during site visits to discuss community priorities PICO to build evaluation question and frame evaluation plan

Implementation Planning C P I Needs Assessment Model Selection Implementation Planning (benchmarks) Evaluation Question O The evaluation question is linked to and informed by the needs assessment and model selection.

What Is a Rigorous Evaluation? Rigor means using an appropriate research design to answer your evaluation question(s) Rigor also means using systematic methods-- keeping as many things as consistent and standard across all aspects of the evaluation as possible Source of info on methodological rigor: DOHVE Evaluation Technical Assistance Brief, May 2011

Traditional ways of knowing Levels of Rigor Experimental Design Quasi-Experimental Design Pre-Post Design Case Studies Opinions, Ideas Life Experience Community Values Traditional ways of knowing Cultural Norms

Balancing Rigor & Feasibility Need to balance goal of rigor with resources available for evaluation Evaluation can be targeted to specific outcomes or components of the intervention Link evaluation questions to benchmarks

Year 1 Evaluation Activities Find the evaluator who is right for your grant Review HomVEE findings and HV research literature Conduct a high quality needs assessment Select a model that will address the priority needs Start thinking about your evaluation question Budget for a rigorous evaluation Evaluators can contribute to the needs assessment, inform & engage community, review literature on HV

Questions?

The Tribal Evaluation Institute is funded by the Office of Planning, research and Evaluation within the Administration for Children and Families. TEI was awarded to James Bell Associates in partnership with the University of Colorado’s Centers for American Indian and Alaska Native Health and Michigan Public Health Institute. For more information, contact the individuals on this slide. The Tribal Home Visiting Evaluation Institute (TEI) is funded by the Office of Planning, Research and Evaluation, Administration for Children and Families, Department of Health and Human Services under contract number HHSP23320095644WC. TEI is funded to provide technical assistance to Tribal Home Visiting grantees on rigorous evaluation, performance measurement, continuous quality improvement, data systems, and ethical dissemination and translation of evaluation findings. TEI1 was awarded to MDRC; James Bell Associates, Inc.; Johns Hopkins Bloomberg School of Public Health, Center for American Indian Health, and University of Colorado School of Public Health, Centers for American Indian and Alaska Native Health. For more information on TEI contact: Nicole Denmark Kate Lyon Federal Project Officer Project Director Office of Planning Research and Evaluation James Bell Associates, Inc. nicole.denmark@acf.hhs.gov lyon@jbassoc.com