CALIFORNIA DEPARTMENT OF EDUCATION Jack O’Connell, State Superintendent of Public Instruction GREG AUSTIN Director, Health & Human Development Program Part 2: Student and Staff Data from the California Healthy Kids Data System Resilience, School Connectedness, & Academic Achievement Gregory Austin, PhD June 23, 2006

JACK O’CONNELL State Superintendent of Public Instruction GREG AUSTIN Director, Health & Human Development Program Overview to the CHKS Student and Staff School Climate surveys and their use as school improvement tools Data on the level of school assets and connectedness in California schools The relationship of CHKS resilience indicators to API and SAT-9 scores and other measures of academic performance and school behavior. Staff perceptions of staff-student relationships and the school learning environment. Presentation

JACK O’CONNELL State Superintendent of Public Instruction GREG AUSTIN Director, Health & Human Development Program Student behaviors linked to achievement School climate and environment Barriers to learning Teacher-student and staff relations School connectedness and motivation to learn Tools for School Improvement Though mandated in compliance with NCLB Title IV requirements, the CHKS data collection system provides a wealth of information from students and staff to inform and guide school improvement efforts. Including: Funded by CDE’s Safe and Healthy Kids Program Office

JACK O’CONNELL State Superintendent of Public Instruction GREG AUSTIN Director, Health & Human Development Program CHKS Student Survey Nation’s largest, most comprehensive local student behavior data system, grades 5-12 (began 1997; mandated fall 2003) Fulfills NCLB Title IV data requirements Designed to: (a) meet multiple data needs of local, regional, and state agencies; (b) reduce survey burden and costs on schools Assess youth needs and guide program decision making Improve youth well-being and achievement

JACK O’CONNELL State Superintendent of Public Instruction GREG AUSTIN Director, Health & Human Development Program Everything is on the Website:

JACK O’CONNELL State Superintendent of Public Instruction GREG AUSTIN Director, Health & Human Development Program Biennial representative district survey Grades 5, 7, 9, & 11, and Continuation Voluntary, anonymous student participation with parental permission Standardized administration procedures and protections Provide results for aggregation into single database Requirements (CDE)

JACK O’CONNELL State Superintendent of Public Instruction GREG AUSTIN Director, Health & Human Development Program Database Size 85+% of districts have valid school-level data Participating districts represent 95% of state enrollment 90% of counties coordinate for representative data

JACK O’CONNELL State Superintendent of Public Instruction GREG AUSTIN Director, Health & Human Development Program Core (Required) – Demographics – School grades and truancy – ATOD Use and Violence – Exercise, Eating, Height/weight, & Asthma Risk Resilience and Youth Development Module (RYDM) – School Connectedness & External Assets in School/Community (Required) – External Assets in Home & Peer Group – Internal Assets Four supplementary modules (tobacco, AOD use/violence, sexual behavior, physical health) Modular Secondary Survey Content

JACK O’CONNELL State Superintendent of Public Instruction GREG AUSTIN Director, Health & Human Development Program Survey Content (continued) Single Elementary covers Core & RYDM Custom modules allow questions of local interest to be added — on any subject –Enable schools to incorporate program evaluation questions to determine what is working! Not just a Survey — A Data Collection System

JACK O’CONNELL State Superintendent of Public Instruction GREG AUSTIN Director, Health & Human Development Program Grades received (1) Classes skipped/cut (1) Transience (1) AOD use at school; AOD problems with school work and behavior (9) Violence perpetration & weapons possession (3) Victimization and harassment (11) Perceived safety (1) School environmental assets (9) School connectedness (5) Core School Indicators (41 Items)

JACK O’CONNELL State Superintendent of Public Instruction GREG AUSTIN Director, Health & Human Development Program CHKS identifies health-related barriers to learning to guide promotion of LEARNING SUPPORTS: The nonacademic resources and instructional strategies that give students the physical, social, emotional, and intellectual support needed to learn. Learning is impaired when students are: Tired or restless Malnourished or sick Stressed or fearful, bullied or abused Under the influence of alcohol or drugs Assessing Barriers to Learning

JACK O’CONNELL State Superintendent of Public Instruction GREG AUSTIN Director, Health & Human Development Program Resiliency In Action Resilience and Youth Development Module (RYDM) Theoretical Framework & Scales RYDM External Assets School Home Community Peers Improved health, social, and academic outcomes Caring Relationships High Expectations Meaningful Participation Safety Love Belonging Respect Mastery Challenge Power Meaning Cooperation Empathy Problem-solving Self-efficacy Self-awareness Goals and aspirations RYDM Internal Assets

JACK O’CONNELL State Superintendent of Public Instruction GREG AUSTIN Director, Health & Human Development Program Secondary School Asset Scales

JACK O’CONNELL State Superintendent of Public Instruction GREG AUSTIN Director, Health & Human Development Program Internal Asset Scales

JACK O’CONNELL State Superintendent of Public Instruction GREG AUSTIN Director, Health & Human Development Program School Connectedness Scale I feel close to people at this school. I am happy to be at this school. I feel like I am part of this school. The teachers at this school treat students fairly. I feel safe in my school. —Derived from the National Longitudinal Study of Adolescent Health. Questions ask “How strongly do you agree or disagree…” Add Health research shows: School connectedness associated with improved grade- point average and lower out-of-school suspension one year later. Most powerful predictor of low risk behavior. Assesses personal feelings about the school, versus perceived environmental assets by School Asset Scales.

JACK O’CONNELL State Superintendent of Public Instruction GREG AUSTIN Director, Health & Human Development Program Staff School Climate Survey Required biennially along with student CHKS in compliance with NCLB All teachers, administrators, staff grades 5 & above (voluntary) Low-cost, online, easy-to-use Short (43 Core items answered by all) Comparability with CHKS student data Key school reform variables System for collecting other data — Add questions Adopted by National Evaluation of Safe Schools Healthy Students Grantees, 2005+

JACK O’CONNELL State Superintendent of Public Instruction GREG AUSTIN Director, Health & Human Development Program SCS Core Content (43 items) Positive Learning Environment (24-item scale) –Staff-student relationships & school connectedness (6 items) –School norms and standards (9 items) –Student behaviors that facilitate learning (9 items) Staff and student safety (9 items) School rules/policies implementation (2 items) Impact student AOD use (3 items) Adequacy health/prevention services (3 items)

JACK O’CONNELL State Superintendent of Public Instruction GREG AUSTIN Director, Health & Human Development Program What SCS Data Will Tell You Is school an inviting and supportive learning environment with high standards? Are students well-prepared, able & motivated to learn? Are students connected to school? Is school a supportive, respectful place to work? Do staff feel responsibility for school improvement? How much is academic achievement a priority? Do staff feel safe? What are the perceived barriers to learning?

JACK O’CONNELL State Superintendent of Public Instruction GREG AUSTIN Director, Health & Human Development Program Total School Assets California RYDM Data

JACK O’CONNELL State Superintendent of Public Instruction GREG AUSTIN Director, Health & Human Development Program % High in Each School Asset California RYDM Data

JACK O’CONNELL State Superintendent of Public Instruction GREG AUSTIN Director, Health & Human Development Program Trends: Total School Assets and 2005 California Student Surveys

JACK O’CONNELL State Superintendent of Public Instruction GREG AUSTIN Director, Health & Human Development Program Trends: High in Three School Assets and 2005 California Student Surveys

JACK O’CONNELL State Superintendent of Public Instruction GREG AUSTIN Director, Health & Human Development Program School Connectedness California RYDM Data

JACK O’CONNELL State Superintendent of Public Instruction GREG AUSTIN Director, Health & Human Development Program Trends in School Connectedness, and 2005 California Student Survey

JACK O’CONNELL State Superintendent of Public Instruction GREG AUSTIN Director, Health & Human Development Program Download at: What are the affects of variations in resilience on annual standardized test scores in California?

JACK O’CONNELL State Superintendent of Public Instruction GREG AUSTIN Director, Health & Human Development Program CDE (via Stuart Foundation) commissioned examination of two questions: Are California students in low performing schools exposed to more health risks and fewer development supports (assets) than students in other schools? (Concurrent or Cross-sectional) How are student health risks and resilience assets related to the progress of California schools in raising test scores? (Longitudinal)

JACK O’CONNELL State Superintendent of Public Instruction GREG AUSTIN Director, Health & Human Development Program CHKS/Test Score Analyses CHKS (combined grades) –Core Module (1,700 schools, 800,000 students) –Resilience Module (600 schools) Academic Performance Index (API) scores — concurrent analyses (Year 1) SAT-9 scores by curriculum area — longitudinal analyses 35 health variables School-level analysis using regression techniques Adjusted for racial/ethnic composition, parental education, ELL students, free/reduced meals, and baseline test scores (when appropriate)

JACK O’CONNELL State Superintendent of Public Instruction GREG AUSTIN Director, Health & Human Development Program Overall Findings Concurrent: Students in low performing (API) schools were characterized by more health risks and fewer external assets than students in higher performing schools — even after accounting for socioeconomic characteristics. –In 75% of measures Longitudinal: SAT-9 test score gains were larger one year later in schools whose students reported high levels of physical well-being, safety, and resilience, and low levels of substance use, violence, and poor mental health — controlling for baseline scores and SES. –In 40% of measures

JACK O’CONNELL State Superintendent of Public Instruction GREG AUSTIN Director, Health & Human Development Program High in Total School Assets and API

JACK O’CONNELL State Superintendent of Public Instruction GREG AUSTIN Director, Health & Human Development Program High in School Caring Relationships and API

JACK O’CONNELL State Superintendent of Public Instruction GREG AUSTIN Director, Health & Human Development Program High in School High Expectations and API

JACK O’CONNELL State Superintendent of Public Instruction GREG AUSTIN Director, Health & Human Development Program High in School Meaningful Participation and API

JACK O’CONNELL State Superintendent of Public Instruction GREG AUSTIN Director, Health & Human Development Program School Caring Relationships and 1-year change in SAT ReadingLanguageMathematics Percent reporting caring relations with adults at school Annual Change in SAT-9 (NPR) Source: California Healthy Kids Survey & STAR data files.

JACK O’CONNELL State Superintendent of Public Instruction GREG AUSTIN Director, Health & Human Development Program School High Expectations and 1-year change in SAT ReadingLanguageMathematics Percent reporting high expectations at school Annual Change in SAT-9 (NPR) Source: California Healthy Kids Survey & STAR data files.

JACK O’CONNELL State Superintendent of Public Instruction GREG AUSTIN Director, Health & Human Development Program Meaningful Participation in Community and 1-year change in SAT ReadingLanguageMathematics Percent in school reporting meaningful participation in community Change in SAT-9 (NPR) Source: California Healthy Kids Survey

JACK O’CONNELL State Superintendent of Public Instruction GREG AUSTIN Director, Health & Human Development Program Methodological Limitations Limited to secondary schools that conducted CHKS before required (less representative) –Especially applies to resilience data, which may be affected by a selection bias. Non-experimental data. –Other unmeasured factors could account for relationship of assets to changes in test scores School-level analysis. –Results need confirmation using student-level data. Did not examine how change in resilience related to change in SAT-9 Still, results indicate that promoting school environmental assets and school connectedness, as measured by the CHKS, may be an important component of any effort to turn around low performing schools and improve test scores.

JACK O’CONNELL State Superintendent of Public Instruction GREG AUSTIN Director, Health & Human Development Program Other Student Data Linking Assets/Connectedness to School Performance Indicators

JACK O’CONNELL State Superintendent of Public Instruction GREG AUSTIN Director, Health & Human Development Program Aggregated State CHKS Data Skipping School/Classes and School Assets “During the past 12 months about how many times did you skip school or cut classes?” 5% 9% 15% 28% 10% 5% 2% 25% 8% 3% 1% 19% 0% 5% 10% 15% 20% 25% 30% 7th9th11thNon Traditional Low Moderate High School Assets % of Students Skipping School or Cutting Classes

JACK O’CONNELL State Superintendent of Public Instruction GREG AUSTIN Director, Health & Human Development Program Aggregated State CHKS Data Poor Grades (D/F) & School Assets “During the past 12 months, how would you describe the grades you mostly received in school? (Mostly Ds & Fs)” School Assets

JACK O’CONNELL State Superintendent of Public Instruction GREG AUSTIN Director, Health & Human Development Program School Connectedness: Low Performing Schools (LP) vs. State Average (Av) CHKS — Lowest API Decile (n=13)

JACK O’CONNELL State Superintendent of Public Instruction GREG AUSTIN Director, Health & Human Development Program Asset Promotion Reduces Involvement in Risk Behaviors that are Barriers to Learning

JACK O’CONNELL State Superintendent of Public Instruction GREG AUSTIN Director, Health & Human Development Program Bringing Weapons to School and School Assets School Assets % of Students Who Have Brought a Weapon to School in the Last 30 Days Aggregated State CHKS Data “During the past 30 days, on how many days did you carry [a knife, a gun, a club, any other weapon] on school property?”

JACK O’CONNELL State Superintendent of Public Instruction GREG AUSTIN Director, Health & Human Development Program Drunk/high at School 2+ Times & School Assets

JACK O’CONNELL State Superintendent of Public Instruction GREG AUSTIN Director, Health & Human Development Program Substance Abuse Affects Learning Environment for All 9% of 9th graders, Heavy Drug Users are responsible for: 21% of school fighting, 24%-27% of school vandalism, D/F’s, and chronic truancy (once a month or more), 34% of weapons possession. 18% of 9th graders, Heavy Drug/Alcohol Users are responsible for: About 30% of fighting and vandalism. About 40% of chronic truancy, D/F’s, and weapons possession.

JACK O’CONNELL State Superintendent of Public Instruction GREG AUSTIN Director, Health & Human Development Program Resilience and Reducing Learning Barriers Resilience promoting strategies targeting youth involved in, or at high rise of, substance abuse and other problem behaviors may not only reduce odds of school failure but improve the learning environment of the whole school. This would be especially important in low performing schools heavily impacted by substance abuse.

JACK O’CONNELL State Superintendent of Public Instruction GREG AUSTIN Director, Health & Human Development Program Staff School Climate Survey: Staff-Student Relations Learning Environment Indicators

JACK O’CONNELL State Superintendent of Public Instruction GREG AUSTIN Director, Health & Human Development Program Do Schools Have a Positive Learning and Working Environment (SCS Scale)? Source: CHKS Staff School Climate Survey, (n=67,464)

JACK O’CONNELL State Superintendent of Public Instruction GREG AUSTIN Director, Health & Human Development Program % High in Learning Environment Subscales Source: CHKS Staff School Climate Survey, (n=67,464)

JACK O’CONNELL State Superintendent of Public Instruction GREG AUSTIN Director, Health & Human Development Program % High on Learning Scales and API Source: CHKS Staff School Climate Survey, (n=67,464)

JACK O’CONNELL State Superintendent of Public Instruction GREG AUSTIN Director, Health & Human Development Program Three Resilience Indicators Source: CHKS Staff School Climate Survey, (n=67,464)

JACK O’CONNELL State Superintendent of Public Instruction GREG AUSTIN Director, Health & Human Development Program Three Learning Indicators % Strongly Agree Source: CHKS Staff School Climate Survey, (n=67,464)

JACK O’CONNELL State Superintendent of Public Instruction GREG AUSTIN Director, Health & Human Development Program Do Staff Feel Students Are Motivated to Learn? Source: CHKS Staff School Climate Survey, (n=67,464)

JACK O’CONNELL State Superintendent of Public Instruction GREG AUSTIN Director, Health & Human Development Program Do Schools Promote Resilience and Assets? Source: CHKS Staff School Climate Survey, (n=67,464)

JACK O’CONNELL State Superintendent of Public Instruction GREG AUSTIN Director, Health & Human Development Program Conclusions The CHKS data on resilience are important for guiding and monitoring school improvement efforts. Youth development and successful learning are complimentary and synergistic processes. Students’ capacity for learning cannot be optimally engaged if their basic developmental needs are not being met. Promoting resilience will help foster school connectedness and reduce involvement in risk behaviors that are barriers to learning, thereby enhancing learning motivation and the likelihood of academic improvement. The greatest need and challenge lie in high schools.

JACK O’CONNELL State Superintendent of Public Instruction GREG AUSTIN Director, Health & Human Development Program A School Climate that Motivates to Achieve… Provides students with supportive, caring connections to adults at school who model and support healthy development. Provides clear and consistent messages that students can and will succeed. Involves students in meaningful activities and decision-making in school

JACK O’CONNELL State Superintendent of Public Instruction GREG AUSTIN Director, Health & Human Development Program Now What? Listening to Students Workshop! CHKS Hotline Bonnie Benard & Carol Burgoa Conducting Focus Groups with Students to Improve Understanding of CHKS Data and How to Promote Positive Student Behavioral, Health, and Academic Outcomes

JACK O’CONNELL State Superintendent of Public Instruction GREG AUSTIN Director, Health & Human Development Program Implementing Resilience/Youth Development Strategies in Our Schools  CHKS Resilience & Youth Development Handbook  Getting Results 5  Resiliency: What We Have Learned  Student Fishbowl/Focus Groups  “Turnaround Teachers and Schools”

JACK O’CONNELL State Superintendent of Public Instruction GREG AUSTIN Director, Health & Human Development Program Listening to Students Michael Fullan, The New Meaning of Educational Change 1991 Educational change, above all, is a people-related phenomenon for each and every individual. Students, even little ones, are people too. Unless they have some meaningful (to them) role in the enterprise, most educational change, indeed most education, will fail. I ask the reader not to think of students as running the school but to entertain the following question: What would happen if we treated the student as someone whose opinion mattered in the introduction and implementation of reform in schools?