COSA Assessment Conference Mickey Garrison Tony Alpert Jon Wiens Beth LaDuca

Need a question about preliminary results! What data are essential to ensuring continuous improvement? (strand 2) Where can administrators find state assessment and report card data? (strand 1) What tools can be used to help focus analysis, planning and evaluation? (strand 2)

3 Broadly Explore Successes & Challenges Winnow Data Infer Cause/Effect Relationships SMART Goals Hypothesize Improvement Strategies Start with OAKS results to narrow the focus for digging deeper at the classroom level. Use student and adult data collected locally to dig deeper and gain clearer insights.

District A District B

Math, , LEP Reading, , Gender

Subtest or Strand scores are based on fewer items Fewer items = greater measurement error = lower reliability of scores Use with caution Understand this limitation Triangulate—look for convergent evidence to develop interventions or instructional plans for students.

leadership/feb09/vol66/num05/Unraveling-Reliability.aspx

Measures of error for group scores tell you how much confidence you can have that you’ve pinpointed the group’s actual performance (provides a range or band of scores) Influenced by size of group and the size of the range of the scores of students in the group Student level—provides a range of values that represent the range within which a student would score again. Influenced by number of items a student answers for a particular score category The more items answered, the smaller the measure of error.

Compare the size of measurement error for total mathematics vs. each of the five subscores

14 Broadly Explore Successes & Challenges Winnow Data Infer Cause/Effect Relationships SMART Goals Hypothesize Improvement Strategies Use simple or complex data tools to collect and organize data and gain insights from your data. Use process tools to winnow data, analyze for meaning, set SMART goals, hypothesize improvement strategies and facilitate decision making.

1. Collect and chart data 2. Analyze strengths and obstacles 3. Establish goals: set, review, revise 4. Select instructional strategies 5. Determine results indicators What impacts effectiveness of data teams and use of data to inform instruction? What roadblocks exist to prohibit effectiveness?

Calcula- tions & Estimation Measurement Statistics & Probability Algebraic Relationships Geometry Grade Grade Grade Grade 3 Grade

Calcula- tions & Estimation Measurement Statistics & Probability Algebraic Relationships Geometry Grade Grade Grade Grade 3 Grade

Vocabulary Read to Perform a Task Demonstrate General Understanding Develop an Inter- pretation Examine Content & Structure – Info. Text Examine Content & Structure – Lit. Text Grade Grade Grade Grade 3 Grade NANA

Vocabulary Read to Perform a Task Demonstrate General Understanding Develop an Inter- pretation Examine Content & Structure – Info. Text Examine Content & Structure – Lit. Text Grade Grade Grade Grade 3 Grade NANA

Structures and learning conditions that precede and predict results. These may include planning time, focused resources, intentional professional development. Factors that have a strong correlation with results. Antecedents (organization) Teacher-student practices that engage students in thinking. These practices require training, practice, and ongoing professional development to implement with mastery. These do not necessarily influence student cognition Instructional Strategies Adult behaviors and routines that create specific effects or results in student behavior or achievement. These may include classroom management routines, classroom schedule routines, transition routines, etc. These do not necessarily require extensive training to implement with mastery. Causes (implementation) Student learning which can be measured by formative, interim, and summative assessments. The measures do not have to be extensive or an additional layer but rather an intentional use of existing materials and resources Effect

Instructional Strategies Marzano, Pickering, Pollock Average Effect Size Percentile Gain Number of Studies Identifying Similarities & Differences Summarizing & Note Taking Reinforcing Effort & Providing Recognition Homework & Practice Nonlinguistic Representation Cooperative Learning Setting Objectives & Providing Feedback Generating & Testing Hypotheses Cues, Questions, & Advance Organizers

Triangulate ! 23

Triangulation!

25 Understanding Antecedents of Excellence Achievement of Results Lucky Losing Ground Leading Learning * Source: The Leadership and Learning Center

What’s Working? Evidence that Convinces Losing Lucky Learning Leading

Mickey Garrison

28