Connecting for Success Middle schools capacity to use early warning data to prevent school dropout P20 Data Exchange Partnership Data Summit October 7, 2014 Tammi Chun Vice President for Programs Tom Kelly VP for Knowledge, Evaluation & Learning

Agenda Introduction 5min Overview of Connecting for Success (CFS) 10min Framework for assessing school capacity to use data 5min Year 1 Findings and Challenges 10min Next steps in Years 2 and 3 5min Questions & discussion 15min Evaluation of session 5min

“Connecting for Success” Pathways to Resilient Communities Goal Vulnerable and at-risk individuals and families in Hawai‘i have access to and receive high-quality and effective services and supports that promote their well-being and helps them thrive. At-risk Youth “Connecting for Success” Vulnerable Families “Housing ASAP” Elderly

Pathways Funder Collaborative Aloha United Way American Savings Bank Atherton Family Foundation Bank of Hawaii Foundation Central Pacific Bank Foundation Cooke Foundation, Ltd. Harold K.L. Castle Foundation Hawaii Community Foundation Hawaii Tobacco Prevention and Control Trust Fund Kosasa Family Fund Omidyar Ohana Fund Richard M. Smart Fund Stupski Family Fund Kresge Foundation McInerny Foundation Pooled funds of $8.1 M (over 3 years) for at-risk youth Twice a year convening to review progress and provide strategic input Shared learning and policy change agenda Site visits Stevenson Middle School

Overall Pathways Approach Invest in Data Capacity & Evaluation to Demonstrate & Learn Invest in Promising Models that Can Achieve Results for Vulnerable Populations Invest in Coordinated Networks of Stakeholders to Create a Change Agenda Invest in Transferring Lessons so Systems Can Scale & Sustain Change

Connecting for Success “Change Statement” IF the capacity and efforts of Hawaii’s public schools— together with their community partners—are strengthened to identify, support and focus effective services on middle school students most at risk of dropping out of school THEN a large cohort of Hawaii’s 6th-8th graders most at risk of dropping out of school will strengthen their connection to school engage in healthy behaviors successfully transition to high school and complete 9th grade on-time SO THAT more students will graduate high school on-time, prepared for success in college, a career and in the community.

15 Connecting for Success 3-year grants reach 10 middle schools Waimanalo Elem. & Inter. and P4C Hawaii (UH) Iao Inter. & Maui Family Support Services Waimea Canyon Middle & Hale ‘Ōpio Kaua‘i, Inc. Stevenson Middle & Kula No Na Po‘e Hawai‘i Ewa Makai Middle Waimea Middle Public Charter Central Middle Waiakea Inter. Kealakehe Inter. & Friends of the Future Pahoa High & Inter.

Schools use HIDOE Early Warning System (in eCSSS) to identify students who are not “on-track” on key indicators Early Warning Indicators Marks (Quarter grades) Attendance Incidents (Behavior)

Connecting for Success reaches a high need group of students School Year 2013-14 Targeted Students in Cohort Middle School Statewide Number of students 931 38,806 Number of schools 10 83 % students low-income 74% 51% % students English Language Learners 14% 8% % students Native Hawaiian 28% % boys 65% Waimea Middle Public Charter School

Grantees are implementing programs that promote the CDC-validated programs and that align with schools’ Academic Plans for student success and school improvement Types of Supports/Interventions: Mentoring Academic tutoring Counseling/student support groups (social skills, study skills, behavioral health) Parent engagement, family education Service learning College/career planning Teacher training (differentiated instruction, guidance/health curriculum) Guidance curriculum High school transition activities Attendance monitoring and interventions Individual learning plans Central Middle School

Grantees meet as a Community of Learners To accelerate learning and provide technical support: Facilitated by Hawaii P-20 (UH) together with Education Delivery Institute (DC-based) Monthly convening during school year: In 2013-14, 4 in-person meetings (including National Drop Out Prevention Conference) plus 6 webinars Year 1 focus: Quality implementation including data use, continuous improvement, effective interventions, project management Year 2 focus: Student attendance, routines for continuous improvement including data collection and use

evaluates CFS rigorously

Year 1 Results for Cohort Students Coursemarks (Grades) Quarter 1 Quarter 4 11% Based on school reports of cohort students’ status on the Hawaii Early Warning System 54% 45% 35% 44% 11% Year-End Results 13% 26% 9% 18% 77% 56% Attendance

Schools Have the Capacity to Access and Use Data Effectively Data accessibility and use Staff member access to & comfort with DOE data systems Data centralization – Ability to combine data from multiple data sources Productive data practices Data skills of school staff – Especially the “data lead” Data reflection routines – School staff routinely review and assess data Staff member access to/comfort with data systems: At least one staff member on the team needs to have access to key databases (including eCSSS and eSIS) and be able to export from these databases on a regular basis. This staff member can then share the exports with the data lead. Data centralization: Schools should combine all their relevant data into one master spreadsheet so they can look across data systems and triangulate results. Data leads’ attributes: Data leads need to have specific attributes, including being fully integrated into the CFS team and having dedicated data management time, in order to support their teams’ data use. Data leads should also be able to transform the data into visuals or other easily comprehensible summaries. Data reflection routines: CFS teams need to meet regularly to reflect on their cohort’s data and use the data when they make decisions.

Year 1 Baseline Assessment FACTORS A B C D E Access to and comfort with data systems yes partly Data centralization unknown no Skills of data lead Data reflection DATA ACCESS & USE PARTLY DATA ROUTINES YES NO Overall   FACTORS F G H I J Access to and comfort with data systems yes no Data centralization unknown Skills of data lead partly Data reflection DATA ACCESS & USE PARTLY YES DATA ROUTINES NO Overall  

Data Accessibility STRENGTH CHALLENGES Ensuring staff access to all relevant DOE data systems eCSSS, eSIS, grades, attendance, formative assessments, discipline, other CHALLENGES Bottleneck in data access Uneven data permissions and log in capabilities Grades I don’t think … for all the other data pieces we look at there’s anybody at the school … that knows where every one of those reports are at except for me. CFS Team Member

Data Use STRENGTH CHALLENGES Schools exporting and combining data into Excel spreadsheets for deeper analysis CHALLENGES DOE data systems not easy to combine or triangulate Labor intensive Excel version and archiving Excel analytic skills Data protection HEWS data doesn’t “freeze” so difficult to look at comparison data over the school year We need to have data to look at. Because in all the years past, I’ve been here for almost 16 years now – we never looked at data. We look at failing grades and attendance and referrals for discipline but never all together. CFS Team Member

Use of HEWS STRENGTH Practice! Requiring the use of HEWS to identify a high-risk cohort of students helped school staff become more familiar and facile with HEWS CHALLENGES Changes to HEWS business rules and definitions Difference in elementary school indicators for incoming 6th graders Quarter grades are not a good leading indicator of academic progress Cohort view of progress on HEWS indicators specific to the user Challenges in exporting HEWS data for analysis

Data Routines STRENGTHS School data leads integrated into school management teams School data leads with good analysis and data visualization skills Changes in data team routines to include review of HEWS data New data teams to review HEWS data CHALLENGES Data leads not integrated or participating in key management / decisionmaking teams Data leads differ in amount of time dedicated No backup or redundancy – data lead is the only person with access and skills

Data Routines I think [we] have always been good at collecting data, but we don't always sit down and look at it.   …I felt that [the counselors] haven't been able to be as effective as they could be. They can't be everywhere, they have [many] kids [they are responsible for]. The data will help them determine where they need to spend the most time, where they need to create connections with the families. CFS Team Members

Other Data Issues STRENGTH CHALLENGES Strong interest and demand for real-time data and ability to aggregate and analyze CHALLENGES No current easy aggregation in DOE systems at school level No data visualization No (easy, consistent) tracking of service participation, dosage, or frequency

Data Culture [The principal] has always been data driven. This isn’t a stretch for us. If anything it is reinforcing our belief and our focus on data. CFS Team Member – Data Lead

Year 2 (2014-2015) & Year 3 (2015-2016) iResult Data coaching Data integration Data analysis Data visualization Service participation, frequency, dosage Data coaching Analysis of attendance and chronic absence Skill enhancement of data leads Peer learning group Next steps