IMPLEMENTING COMPLEX EVIDENCE- BASED PROGRAMS FROM SERVICE TO SCIENCE AND BACK AGAIN REAP Conference Santa Fe, NM March 20, 2008.

Slides:



Advertisements
Similar presentations
Research Strategies: Joining Deaf Educators Together Deaf Education Virtual Topical Seminars Donna M. Mertens Gallaudet University October 19, 2004.
Advertisements

Building Team Connections in a Health Sciences Library Using the DiSC® Profile Nancy Allee, AHIP, Deputy Director, and Jane Blumenthal, AHIP, Director,
A Review of Published Community-Based Participatory Research (CBPR) & Recommendations for Future Scholarly Work Darius Tandon, PhD Assistant Professor,
Communities of Practice in Child and Youth Mental Health
Evidence Based Practices Lars Olsen, Director of Treatment and Intervention Programs Maine Department of Corrections September 4, 2008.
SCHOOL COUNSELING Fran Hensley, M.A.Ed. School Counselor
National Indian Youth Leadership Project McClellan Hall, Founder.
PBIS TIER 1: FAMILY PARTNERSHIP STRATEGIES
Parent Connectors: An Evidence-based Peer-to-Peer Support Program Albert J. Duchnowski, Ph.D. Krista Kutash, Ph.D. University of South Florida Federation.
1 Strengthening Teaching and Learning: Educational Leadership and Professional Standards SABES Directors’ Institute July 2011.
Peering into the black box between pre and post: A theory of change related to interactive/ experiential delivery methods Susan Carter Evaluation Coordinator.
Project Venture A project of: National Indian Youth Leadership Project.
Karen L. Mapp, Ed.D. Deputy Superintendent, Boston Public Schools
Behavior Domain, Behavior Determinants and Behavior Change Health Behavior: CHAPTER 21.
Reducing Disproportionality in Suspensions and Positive Behavior Supports Suzann Wilson Lori Rogerson.
Principles of High Quality Assessment
The APPA Leadership Institute
Community Planning Training 1-1. Community Plan Implementation Training 1- Community Planning Training 1-3.
LONG BEACH YMCA YOUTH INSTITUTE PRESENTATION. WHAT IS A YOUTH INSTITUTE? History Goals Nine Principals Youth Development Foundation Recruitment Technology.
Solid Catalysts – Synthesis and Characterisation Techniques Emmanuel Iro.
School’s Cool in Childcare Settings
Chapter 2 Ensuring Progress in the General Curriculum Through Universal Design for Learning and Inclusion Each Power Point presentation can be viewed as.
SW-PBS District Administration Team Orientation
Comer School Koala EmilyMelodySelinaAmiee. Comer School Koala(Wang Zhen) Emily(Jiang Yan)Melody (Li Xiuhua) Selina(Tong Lingtiao)Amiee(Li Minlim)
Intro to Positive Behavior Interventions & Supports (PBiS)
School’s Cool in Kindergarten for the Kindergarten Teacher School’s Cool Makes a Difference!
SELECTED RESULTS FROM THE MINORITY EXECUTIVE LEADERSHIP INITIATIVE SURVEY Kayla Jackson National Network for Youth Exploring Racism’s Impact on Youth Homelessness.
Filling evidence gaps Karen Glanz, PhD, MPH Emory University Glorian Sorensen, PhD, MPH Harvard University Dana-Farber Cancer Institute.
Peoria Unified World Languages and Immersion Programs Dr. Heather Cruz March 25, 2014.
Creating Lasting Family Connections. Program Funding The Creating Lasting Family Connections program is funded by Title V and the Indiana Criminal Justice.
Sustainability Training Series 2015 From Piloting to Sustaining Practices August 3, :00pm - 3:00pm The Essentials of Full Implementation Part I:
California Statewide Prevention and Early Intervention (PEI) Projects Overview May 20, 2010.
A Framework for Making a Difference Rob Horner, University of Oregon Deputy Director of the Research to Practice Division for the U.S. Department of Education’s.
A System of Care for Children’s Mental Health: Expanding the Research Base Using the System of Care Practice Review (SOCPR) to Facilitate System Transformation.
Nef (the new economics foundation) Co-producing Lambeth what’s possible? Lucie Stephens and Julia Slay nef, October 2011.
Inquiry and Investigation. What was the TOPIC? PROBLEM? CIVIC INQUIRY?
1 Sandy Keenan TA Partnership for Child and Family Mental Health(SOC) National Center for Mental Health Promotion and Youth Violence Prevention(SSHS/PL)
Bloom’s Taxonomy. It was established in 1956 by Benjamin Bloom Image courtesy of
Defending Childhood Protect Heal Thrive January 25-27, 2011 Sandra Spencer Executive Director National Federation of Families for Children’s Mental Health.
1 The Web of Life Study A program evaluation study of an experiential, positive youth development approach to prevent teen pregnancy for Native youth.
College of Education Helping Schools Evaluate Needs and Select Best Practices in Childhood Mental Health.
: The National Center at EDC
Introduction to School-wide Positive Behavior Support.
Welcome to CE420 Seminar 2 We will begin class at the top of the hour and tonight we will begin an in-depth exploration of varied curriculum models.
RTI International RTI International is a trade name of Research Triangle Institute. Evaluation of the Office of Juvenile Justice and Delinquency.
Dissemination in Canada CICAD Guidelines for School-based Prevention of Substance Abuse VII Meeting of the Expert Group on Demand Reduction September 13,
Common Core Parenting: Best Practice Strategies to Support Student Success Core Components: Successful Models Patty Bunker National Director Parenting.
Instructional Leadership: Planning Rigorous Curriculum (What is Rigorous Curriculum?)
1 A Multi Level Approach to Implementation of the National CLAS Standards: Theme 1 Governance, Leadership & Workforce P. Qasimah Boston, Dr.Ph Florida.
Comprehensive Youth Services Assessment and Plan February 21, 2014.
A needs assessment to inform the development of a behavioral health careers program for minority high school students Scyatta A. Wallace, PhD SUNY Downstate.
An Enhanced Support Model to Promote Mental and Behavioral Wellness among Students in the Avon Public Schools Kelly Grant Director of Pupil Services.
Addressing Unhealthy Substance Use with Older Adults Dawn Matchett,LICSW Hearth, Inc. October 20, 2014.
First Things First Grantee Overview.
Communities In Schools MOSPA Conference May 2017
Proctor’s Implementation Outcomes
Clinical Practice evaluations and Performance Review
a New Focus for External Validity
Interprofessional Student-Run Free Clinic for the Homeless
Overview – Guide to Developing Safety Improvement Plan
Overview – Guide to Developing Safety Improvement Plan
What barriers and facilitators influence the implementation of new high-risk medicine services in Scottish community pharmacies? Ms Natalie Weir1, Dr Rosemary.
FIRST PLACEMENT IS THE RIGHT PLACEMENT
School’s Cool Makes a Difference!
Exploring High Quality Student Affairs Learning Outcomes Assessment Practices at Three American Research Universities Adam S. Green, Ed.D. West Virginia.
The Success of IPS in Oklahoma
IP&B Student Service and Instructional Support Retreat
Partnership Alignment, Collaboration and Continuous Improvement
Linda Mayo Willis and Carolyn Pope Edwards
National Community of Practice on Transition Past, Present and Future
Presentation transcript:

IMPLEMENTING COMPLEX EVIDENCE- BASED PROGRAMS FROM SERVICE TO SCIENCE AND BACK AGAIN REAP Conference Santa Fe, NM March 20, 2008

STUDY DETAILS RESEARCH TEAM Susan Carter, PI Gladys Levis-Pilz Heather Yazzie Support provided by a FIRE grant from the Office of Juvenile Justice and Delinquency Prevention, US DOJ

THE PROBLEM We are faced with the paradox of non- evidence-based implementation of evidence- based programs (Drake, Gorman, & Torrey, 2002) Great emphasis on identifying evidence-based programs, but little attention paid to their implementation beyond fidelity discussions, especially for complex programs

RESEARCH QUESTIONS (After Fixen et al, NIRN, 2005) BASED ON STAGES OF PROGRAM IMPLEMENTATION Exploration and Adoption Program Installation Initial Implementation Full Operation Innovation and Sustainability

DESIGN AND METHODS BASED ON NIYLPS PROJECT VENTURE Created by McClellan Hall, Founder of National Indian Youth Leadership Project, Gallup, NM Grew out of camp program into year round model in 1990 Named by NREPP as a Model Program in 2004 Re-reviewed by NREPP, designated evidence-based program in 2007 Universal prevention model based on experiential, youth development principles for Native American mid-school youth

Project Venture 3 levels of non-negotiables Population served Program components Positive approach Outdoor adventure and experiential challenge –Experiential education –Wilderness outdoor adventure –Physical education –Nature challenge Service learning –Active learning –Academics in action –School or community service –Parent involvement Native culture based leadership –Culture and values –Language and history –Cross-cultural exchange and awareness –Elders Program delivery 200 hours of contact annually Risk management principles followed, including sequential progression of elements including –Physical safety –Mental health safety Youth to follow four levels of progression through program –In school (highly recommended) –After school –Weekend and overnight –Intensive camp experience Outcomes measured Focus on Native youth Middle school aged youth* High school aged Service Staff Mix of risk levels (program should not have stigma of prevention program)* Bold: Basic non-negotiable (element must be in place, even from beginning) Regular: Phased-in non-negotiable (element must be in place, but may be phased in over time) Italic: Recommendation (element is recommended, but not required) *:Modification allowed (element may differ if program is appropriately modified)

DESIGN AND METHODS Phase 1: In-depth Case Studies of a few PV programs –Identify all known PV adoption sites in US –Select 8 diverse sites considering rural/urban, new/experienced agency, agency type, etc. –Send invitation letter –Conduct on-site program observations –Conduct semi-structured interviews –Review documents related to implementation –Complete PV Replication Instrument –Complete saw/read/heard about it table

8 Project Venture: currently ~36 replication sites = PV replication site There are approximately 36 replication sites across 17 states

Interview Guide Questions Exploration and Adoption How did you hear about PV in the first place? How/why was the decision made to select? What are the components of your program? Is the programming running fully or partially? Community Support? School Support? Who makes the decisions for PV? Is it important that your facilitators be American Indian? Program Installation How did you prepare to implement PV? Who made those decisions? Staffing? Financing? Restructuring? What about now, how is PV implemented? Staffing? Financing? Restructuring? Initial Implementation Participants, who? How are they selected? Risk levels? PV components: which?, how configured? Full Operation How did you implement PV components? Did you begin with all components or phase them in? Who benefits? How do you feel about PV? What kinds of out comes have you had? Are you measuring outcomes? Tell me about this. Would you share data with NIYLP? Do you use it to improve programs? Does your program hold true to the vision of PV? (go through Program Implementation instrument with them) Innovation and Sustainability How do you manage change in population, financing, community political/tribal, drift/ fidelity? Any plans to sustain? How?

DESIGN AND METHODS Phase 2: Broad study of all PV programs –Develop survey based on Phase 1 findings –Administer via web, mail, phone to all known PV sites

PRELIMINARY FINDINGS FROM CASE STUDIES From earlier Project Venture studies, Blooms Taxonomy seemed a useful framework upon which to hang observational data Life lessons available to individual youth increased through exposure to other facilitators and participants and these lessons also promoted higher levels of cognitive and affective learning (as detailed below).

CASE STUDY FINDINGS (CONT.) The researchers attention kept returning to an idea of evolution or levels resulting from provider-participant and participant- participant interactions. These thoughts in turn led to a review of the educational psychologist Benjamin Blooms taxonomy of levels of learning (Bloom, B. (ed.) Taxonomy of Educational Objectives: The Classification of Educational Goals,1956, New York: Longmans Green).

CASE STUDY FINDINGS (CONT.) This classic theory provides a classification of levels of thinking and feeling behaviors thought to be important in the process of learning. The taxonomies are both descriptive and prescriptive as it orders categories of cognitive, affective, and psychomotor skills. From basic/low levels to ever more abstract/higher levels of learning.

While experimental studies are crucial for advancing our understanding of what works in adventure programming, the qualitative perspective has the power to show us how programming works. Intentionally designed and executed qualitative research enriches our understanding of what goes on in the black box between pre and post tests and provides us with the stories to tell and prescriptions for action.

Blooms Taxonomy: A Promising Analytic Framework Cognitive Skills (from less to more complex) Knowledge Comprehension Application Analysis Synthesis Evaluation

Blooms (cont.) Affective Skills (from less to more complex) Receiving phenomena Responding to phenomena Valuing Organization Internalizing values