A Collaborative Mixed-Method Evaluation of a Multi-Site Student Behavior, School Culture, and Climate Program.

Slides:



Advertisements
Similar presentations
WV High Quality Standards for Schools
Advertisements

Progress Towards Reading Success: The Reading First Evaluation Prepared by: Amy Kemp, Ph.D. Research Associate and Patricia A. Muller, Ph.D. Associate.
Through Instructional Rounds
When Change Runneth Over: Lessons Learned From Implementing Multiple Change Projects Simultaneously PDN Presentation July 2013.
Multidisciplinary Teams
Family Resource Center Association January 2015 Quarterly Meeting.
1 Why is the Core important? To set high expectations – for all students – for educators To attend to the learning needs of students To break through the.
Beth Rous University of Kentucky Working With Multiple Agencies to Plan And Implement Effective Transitions For Head Start Children Beth Rous University.
Talbert House Project PASS Goals and Outcomes.
Quality evaluation and improvement for Internal Audit
Parent Introduction to School-wide Positive Behavior Supports (SW-PBS)
Early Childhood Mental Health Consultants Early Childhood Consultation Partnership® Funded and Supported by Connecticut’s Department of Children and Families.
School’s Cool in Childcare Settings
TIMELESS LEARNING POLICY & PRACTICE. JD HOYE President National Academy Foundation.
Bracknell Forest Council: Evaluation of the Domestic Abuse Perpetrator Service (DAPS) Liz Phillips.
Essential Concepts for School for Prevention First Patricia Berry, M.A. CSAP Essential Concepts for School m 1 BUILDING THE 5 ESSENTIAL.
Shared Decision Making: Moving Forward Together
School Governance for Parents: Advisory Committees How ELAC and SAC Can Help Your School Develop a Better Balanced Scorecard.
New PBIS Coaches Meeting September 2,  Gain knowledge about coaching  Acquire tips for effective coaching  Learn strategies to enhance coaching.
The CIS Model Research, Rationale, & a Business Plan.
1. 2 Why is the Core important? To set high expectations –for all students –for educators To attend to the learning needs of students To break through.
Families as Partners in Learning Principals and teaching staff Why are partnerships important?
Intro to Positive Behavior Interventions & Supports (PBiS)
School’s Cool in Kindergarten for the Kindergarten Teacher School’s Cool Makes a Difference!
Better Deal for Business Presentation to LSC West Yorkshire Skills Team Pat Lister Better Deal for Business Officer at Yorkshire Forward.
Student Support Team (SST) Training A Humanware Strategic Plan Activity Cleveland Metropolitan School District 1.
April 2005-IOM1 SSA/AUCD: A National Collaboration.
Assistant Principal Meeting August 28, :00am to 12:00pm.
Fundamentals of Evaluation for Public Health Programs ROBERT FOLEY, M.ED. NIHB TRIBAL PUBLIC HEALTH SUMMIT MARCH 31,
Objectives 1. Children will be supported in an integrated way through the establishment of a Start Right Community Wrap- Around Programme in the target.
Chapter 11 Referring and Evaluating for Special Education.
Intro to Positive Behavior Supports (PBiS) Vermont Family Network March 2010.
Promoting a Culture of Evidence and Use of Data through Program Evaluation Session Theme 2 Presentation to: OSEP Project Directors’ Conference July 21,
Page 1 Fall, 2010 Regional Cross Sector Meeting Elements of an Effective Protocol.
City Year: Putting Idealism to Work for Philadelphia Students
1 CCR Conference Summer 2014 Building Capacity of Content Teachers through A Comprehensive Literacy Initiative.
The Wisconsin RtI Center (CFDA #84.027) acknowledges the support of the Wisconsin Department of Public Instruction in the development of this presentation.
FLORIDABROWARD ECHOSFAIRECHOSFAIR Not Yet Demonstrating 9%4%12%3% Emerging/Progressi ng 38%26%53%23% Demonstrating53%70%35%74% 2012 Third GradeFLORIDABROWARD.
NAZ as a Promise Neighborhood….. Where opportunities rise to meet their promising future! NAZ Family Academy Graduates.
1. Housekeeping Items June 8 th and 9 th put on calendar for 2 nd round of Iowa Core ***Shenandoah participants*** Module 6 training on March 24 th will.
Guidance for Completing Interim Report I Evaluation Webinar Series 3 Dec 2013.
School-Wide Positive Behavioral Interventions and Supports: Administrator’s Role Donna Morelli Cynthia Zingler Education Specialists Positive Behavioral.
Tier 2/ Tier 3 Planning for Sustainability Rachel Saladis WI PBIS Network/Wi RtI Center Katrina Krych Sun Prairie Area School District.
What, Why and How? IPSCAN Master Class September 2013.
1 Module L R ole of Coaches Coaches’ Monthly Meeting Add DC Name Here.
Background In , City Year Greater Philadelphia (CYGP) provided academic and behavioral supports for students in grades 6-9. Supports include one-on-one.
Buffing Up Use of Assets Ideas for using 1% central set aside.
Data Report July Collect and analyze RtI data Determine effectiveness of RtI in South Dakota in Guide.
CHAPTER 3 Strategy Cards for: Communication with Colleagues, Students, & Families.
Restorative Justice Danielle Paul Restorative Justice Teacher Adviser
School-Wide Positive Behavioral Interventions and Supports: District Coaches’ Meeting Donna Morelli Cynthia Zingler Education Specialists Positive Behavioral.
Annie McLaughlin, M.T. Carol Davis, Ed.D. University of Washington
Broward County Public Schools BP #3 Optimal Relationships
Common Core Parenting: Best Practice Strategies to Support Student Success Core Components: Successful Models Patty Bunker National Director Parenting.
Overview of Student Learning Objectives (SLOs) for
PLEASANTON UNIFIED SCHOOL DISTRICT SCHOOL CLIMATE TRANSFORMATION GRANT.
Integrating the MTSS Framework at the Secondary Level Dr. Jayna Jenkins, Learning and Development Facilitator, MTSS Shelly Dickinson, MTSS Trainer Charles.
Leadership Teams Implementing PBIS Module 14. Objectives Define role and function of PBIS Leadership Teams Define Leadership Team’s impact on PBIS implementation.
CALIFORNIA DEPARTMENT OF EDUCATION Tom Torlakson, State Superintendent of Public Instruction March 5, 2015 California County Offices of Education Attendance.
The PDA Center is funded by the US Department of Education Office of Special Education Programs Stories from the Field and from our Consumers Building.
NYU Child Study Center: Bridges Program Caring Across Communities: Annual Grantee Meeting April
... for our health Building WREN’s Capacity through Strengthening Relationships with Full Support Practices Katherine B. Pronschinske, MT(ASCP)
Laying the Foundation A Discussion on Moving Fidelity of Implementation from Compliance to Capacity Building Carol K. McElvain American Institutes for.
School-wide Positive Behavioral Interventions and Supports District-wide Implementation: Ensuring Success Kentucky Center for Instructional Discipline.
Building Evidence of Effectiveness
BREAKING BARRIERS West Contra Costa Unified School District
B Team Lead Orientation Strengthening Success Team Lead Capacity
Clinical Practice evaluations and Performance Review
Gary Carlin, CFN 603 September, 2012
School’s Cool Makes a Difference!
Presentation transcript:

A Collaborative Mixed-Method Evaluation of a Multi-Site Student Behavior, School Culture, and Climate Program

Tailoring our Evaluation Practices to Leverage Available Resources

Outline What is City Year? Behavior, Culture, and Climate Initiative (BCCI) BCCI Study – Data and methods – Study Challenges Multi-Participant Approach – Benefits – Challenges – Strategies

WHAT IS CITY YEAR? City Year is an education-focused nonprofit organization that partners with public schools to keep students in school and on track to succeed.

25 locations 242 schools 2,600+ corps members

Whole School, Whole Child A poor Attendance B disruptive Behavior C Course failure in math or English

Full time 1-on-1 Small groups After school

9 9 Cities Schools in Pilot Behavior, Culture, and Climate Initiative 8 Supporting student social emotional learning (SEL) skills is strongly connected to work in providing academic supports. Elements of BCCI include: In-class/homeroom behavior support Targeted support: 50 Acts Program Integration of City Year culture Additional training and support for corps members : third year of pilot program and evaluation

Behavior, Culture, and Climate Initiative Third-Year Study 9

BCCI Study Purpose To strengthen implementation and replicate this work across the City Year network. Help the organization understand and share some early outcomes. Guiding evaluation questions centered on outcomes, implementation, replication and evaluation practices.

The Study Evaluation Team External Evaluator: – Brett Consulting Group Internal Staff, members from the: – City Year Headquarters Evaluation Team, and – Headquarters Program Design Team

Data and Methods: Specific to the BCCI student prosocial behaviors rich case observations interviews/focus groups student suspensions, detentions, office referrals

City Year centrally collects network-wide survey and intervention tracking data Relevant network data was used in combination with information collected for the BCCI pilot Internal database tracking student-level interventions Online, network-wide teacher, principal/ school liaison, and corps member surveys

How We Used Evaluation Findings External Engage champions. Inform funders. Attract new funders. Internal Refine program design, metrics, and quality of implementation. Early knowledge provided a head start on planning for the following school year.

Challenges to This Study Variations in program implementations Different ways of measuring same construct by different organizations in the study Data availability, including timeliness Imperfect match of measures with goals of program

Multi-Participant Approach to Evaluation

Benefits of Multi-Participant Approach Better use of available resources Cost and time savings. Increased capacity for access to schools and program staff. Increased capacity for data collection, analysis, and report writing. Ability to draw upon multiple perspectives and backgrounds. Further internal understanding of program Real time information on measurement and outcomes to inform planning for the following school year.

Possible Challenges to the Multi-Participant Approach buy-in by participants objectivity by internal staff openness to approach by external evaluator agreement about where to put energy and resources Lack of… coordinating schedules (esp. for observation) multiple report writers Logistics

Strategies When Working with an External Evaluator in a Multi-Participant Evaluation Choose an external evaluator who… embraces and has experience with this approach Assure that the external evaluator is familiar with… the program model all existing instrumentation/ data collection systems Use the external evaluator to… design at least one key part of the study be involved with overall design

General Strategies for Multi-Participant Approaches clearly define purposes of the study clarify each person’s role reinforce expectations, importance, timelines use point people across multiple sites Communication have one person oversee and coordinate report writing allow time for the unexpected Logistics

General Strategies for Multi-Participant Approaches (2) develop efficient systems to decrease burden on staff and others develop written protocols for data collection train staff and others in data collection protocols Working with Staff emphasize recording of specific behaviors double up on observations share perceptions across observers ask for evidence to back up observations Conducting Observations

Thank You! Contact Us: – Belle Brett: – City Year Evaluation: