Data Warehousing How do we use the data now that we have it? David Rutherford Director, Instructional Support Services ONC BOCES May 18, 2007

Where Are We Going? - Data Analysis in the ONC BOCES Region - Using the 3-8 Assessments for School Improvement - The Two Holy Grails of the 3-8 Assessments

Data Analysis at ONC BOCES Our RIC – South Central RIC, Binghamton, NY Data Analysis Service – Dr. Carol Dean Monthly Meetings – Carol, Instructional Support, Staff Developers

School Improvement and Assessments "not everything that counts can be counted-- not everything that can be counted counts“ -Albert Einstein

School Improvement and Assessments Over-reliance on the 3-8 Assessments

School Improvement and Assessments Limited Nature of the 3-8 Assessments Short Assessment – not everything is assessed Turn-around time – cannot be used effectively as a formative assessment

School Improvement and Assessments

The Two Holy Grails of Assessment Holy Grail Number 1 The State Assessments are verifications of existing assessments within the school/district

The Two Holy Grails of Assessment Holy Grail Number 1 Professional Learning Communities

The Two Holy Grails of Assessment Professional Learning Communities Regional ONC BOCES Plan Communal Organization of Teachers Common Understanding of Standards at Grade Level Development of Common Formative Assessments Collective Evaluation of Results Change in Classroom Practice

The Two Holy Grails of Assessment Coleman et al. (1966) “Equality of Educational Opportunity in the United States.” Studied 600,000 children at 4,000 schools An act of Congress during the civil rights Era. Had the resources, authority and visibility of the federal government behind the study The finding, neatly summarized, meshed with what many people saw with their own observations The credibility, congruence and capacity of the research gave it immense staying power – to this day with the general public

The Two Holy Grails of Assessment Coleman – cont. “[S]chools bring little influence to bear upon a child’s achievement that is independent of his background and general social context” (Coleman et al., 1966, p.325).

The Two Holy Grails of Assessment Holy Grail Number 2 The State Assessments can be used to show growth, giving a better picture of school quality and over time-improvement

The Two Holy Grails of Assessment Holy Grail Number 2 – It’s Coming 1. NCLB Reauthorization 2. Article VII (Contract For Excellence)

The Two Holy Grails of Assessment Holy Grail Number 2 – Achievement vs. Progress Achievement model: Determine whether an increasing percentage of students are gaining proficiency over time. Growth model: Determine how much progress groups of students are making over time.

The Two Holy Grails of Assessment Achievement vs. Progress: The Power of Two Achievement – Measures a student’s performance at a single point in time Progress – Measures a student’s progress between two points in time

The Two Holy Grails of Assessment Achievement vs. Progress: The Power of Two Achievement – Relates to a student’s family background Progress – Not related to a student’s family background

The Two Holy Grails of Assessment Achievement vs. Progress: The Power of Two Achievement – Compares students’ performance to a standard Progress – Compares students’ performance to their own prior performance

The Two Holy Grails of Assessment Achievement vs. Progress: The Power of Two Achievement – Critical to a student’s post secondary opportunities Progress – Critical to ensuring a student’s future academic success

The Two Holy Grails of Assessment Holy Grail Number 2 – Achievement vs. Progress Achievement model: Determine whether an increasing percentage of students are gaining proficiency over time. Growth model: Determine how much progress groups of students are making over time.

The Two Holy Grails of Assessment Holy Grail Number 2 – Growth and the 3-8 Assessments Can we show growth now that we have longitudinal data? Problematic – why?

The Two Holy Grails of Assessment Holy Grail Number 2 – Vertically Moderated vs. Vertically Scaled Vertically Moderated – Meeting the “cut point” at one grade level indicates that the student is prepared for instruction at the next grade level. ** Student progress is measured from grade-to-grade relative to proficiency in meeting the standards

The Two Holy Grails of Assessment Holy Grail Number 2 – Vertically Moderated vs. Vertically Scaled Vertically Scaled – Meeting the same “cut point” in two years is an indication of one full year of growth

The Two Holy Grails of Assessment Holy Grail Number 2 – Vertically Moderated vs. Vertically Scaled NYS Assessments are Vertically Moderated Growth Model requires vertical scaling

The Two Holy Grails of Assessment Holy Grail Number 2 – Growth vs. Value Added Growth models generally refer to models of school accountability that assess the progress of a cohort of students over time with the intent of measuring the progress these students have made (This year’s fourth graders compared to their performance last year as third graders.) Value added models generally refer to a specific type of growth model in which student demographic data or other statistical controls are used to attempt to analyze the specific effects of a particular school, program, or teacher on student learning

The Two Holy Grails of Assessment Holy Grail Number 2 – Achievement vs. Progress Recap Achievement - Where students need to be at the end of P-12 Growth - How they get there

The Two Holy Grails of Assessment Value Added – SAS and Battelle for Kids SAS – Dr. Bill Sanders developed the Value- Added Model Battelle for Kids – Non-profit responsible for VA Professional Development Capital Region and ONC BOCES – Grant to Pilot VA in New York State

School Improvement and Assessments More Information

School Improvement and Assessments "not everything that counts can be counted-- not everything that can be counted counts“ -Albert Einstein