How the CAP Science and Social Studies Tests Measure Student Growth For 2016-17 CMSD

It Begins With How The CAP Tests Were Designed Custom test blueprints developed to clearly define test content and the content balance. Blueprints for the Pretest and Posttest forms within a subject/grade are the same. Ohio teachers wrote new items directly aligned to Ohio’s science and social studies standards and content statements to match the blueprints. All items field-tested with Ohio students. Field test design placed all items within a subject/grade level on same difficulty scale. Field test results guided the construction of final Pretest and Posttest forms. Final forms are aligned to common blueprint and content specifications. This allows direct comparison of Pretest to Posttest performance.

Measuring Student Growth Pretest and Posttest performance can be placed on the same underlying performance scale.

Measuring Student Growth Student growth is the difference on the underlying scale between student performance on the Pretest and student performance on the Posttest.

Measuring Student Growth Because the Pretest is shorter than the Posttest, the growth in raw score from Pretest to Posttest is only a general indicator of real student growth. CAP uses student growth on the underlying performance scale as the best measure of student growth. This scale provides the most precise measure of student growth.

Measuring Student Growth To place a student’s score on the underlying performance scale, SME converts raw scores to performance scale scores.

Measuring Student Growth Student growth is computed by first subtracting the Pretest scale score from the Posttest scale score.

Measuring Student Growth The “Effect Size” statistic is used to add meaning to the student growth score. Calculated by dividing the student growth score by the average standard deviation of the two tests.

Measuring Student Growth The reference point for the statistic is the Local District’s mean and standard deviation on the pretest and the posttest. Effect size data have a long history of use in education and can be used to set meaningful standards for student growth.

Measuring Student Growth An example: Mr. Johnson is one of a number of teachers who teach Grade 6 Social Studies in District A. He has 100 students across all his sections.

Measuring Student Growth Step 1: Grade 6 Social Studies Pretest and Posttest scores for all students in District A are recorded. SME finds the mean and standard deviation for both tests.

Measuring Student Growth Step 2: SME uses established formulas to convert all raw scores to scale scores and computes the student growth on the underlying performance scale.

Measuring Student Growth Step 3: Using District A’s overall grade 6 social studies performance data, SME computes the district’s Average Standard Deviation and converts student growth to an “Effect Size.” For Mr. Johnson’s class, the Effect Size equation for Student 1 would be:

Measuring Student Growth SME computes an effect size for each student:

Measuring Student Growth Step 4: Teachers are assigned an effectiveness category based on the average effect size achieved by their students within a subject/grade level. Average Social Studies Effect Size Social Studies Rating >=1.6700 5 1.0700 - 1.6699 4 0.4700 -1.0699 3 0.1100 - 0.4699 2 < 0.1100 1 Average SCIENCE Effect Size SCIENCE Rating >=1.7800 5 1.1800 - 1.7799 4 0.5800 - 1.1799 3 0.1600 - 0.5799 2 < 0.1600 1