1. Achievement Analyses – Matched Cohort Groups Oklahoma A+ Schools® vs. Randomly Matched OKCPS Students  OKLAHOMA CITY PUBLIC SCHOOLS  PLANNING, RESEARCH, and EVALUATION  413 N.W. 12th St., Oklahoma City, OK 73103 Question: Do students in OKCPS A+ sites outperform their counterparts of equal achievement ability who attend other OKCPS schools across the district? George H. Kimball, Ph.D. Director Selected Highlights of Report

2. During the 2004-05 school year there were six A+ school sites in the Oklahoma City Public Schools. Shown below are the sites for which this analysis was conducted (and the school year in which they implemented the A+ program.)  Britton Elementary (2003-04)  Cleveland Elementary (2002-03)  Linwood Elementary (2002-03)  Mark Twain Elementary (2002-03)  Sequoyah Elementary (2002-03)  Van Buren Elementary (2004-05)  Western Village Academy Charter (2002-03)

3. Methodology  Simple direct comparisons of average scores of one school versus another are of marginal utility in addressing questions of program effectiveness or outcomes.  The more appropriate model for looking at achievement growth across a school year is to produce an equivalent control group for comparison with students in the targeted sites.  Two groups of students are identically matched on achievement ability at the start of the school year.  The matched cohorts are identical in race, free lunch status, sex, grade, special Ed, and LEP status.  Each group receives some “educational treatment” during the course of the school year.  Measure and test the group outcomes at the end of the year to assess between-group differences.

4. Basis for Analysis  Spring 2005 achievement scores from the State of Oklahoma Criterion Referenced Tests (CRT’s), formally called the Oklahoma Core Curriculum Tests (OCC), from students in grades 3 through 5 attending the above sites were compared to randomly matched samples of cohort students selected from OKCPS sites from across the district. Paired samples t-tests were employed to evaluate the statistical significance of differences between the scores of students in the A+ sites and those of their matched cohorts.

5. Students in the A+ schools significantly outperformed their matched cohort comparison group on the reading scores of the spring, 2005 Oklahoma Core Curriculum tests on the overall Oklahoma Performance Index Scores. (Oklahoma City Public Schools, 2006)

6. Students in the A+ schools outperformed their matched cohort comparison group on the reading scores of the spring, 2005 Oklahoma Core Curriculum tests (70% as compared to 66% overall), but the overall difference was not statistically significant. (Oklahoma City Public Schools, 2006)

7. Students in the A+ schools significantly outperformed their matched comparison control group on the mathematics scores of the spring, 2005 Oklahoma Core Curriculum tests on the overall Oklahoma Performance Index Scores.

8. Students in the A+ schools significantly outperformed their matched cohort comparison group on the mathematics scores of the spring, 2005 Oklahoma Core Curriculum tests (72% as compared to 65% overall) on the percent of students scoring satisfactory and advanced. (Oklahoma City Public Schools, 2006)

9. Achievement Summary Oklahoma Performance Index Scores (OPI)  Students in the A+ schools significantly outperformed their matched cohort comparison group on the state OPI reading scores of the spring, 2005 Oklahoma Core Curriculum Tests. Within the A+ schools there was some variability. Two of the seven A+ schools did not outperform their matched cohort comparison group, although only one was statistically significant. Five of the seven did outperform their matched cohort comparison group and three of the five were statistically significant. The between-group effect size at one school was notable in that the +.42 effect size means that their students achieved almost a half a standard deviation better than their matched counterparts.  In mathematics, students in the A+ schools again significantly outperformed their matched comparison group on the math OPI scores of the spring, 2005 Oklahoma Core Curriculum tests. Unlike in reading, all A+ schools outperformed their matched cohort comparison groups in mathematics.  All seven A+ schools made Adequate Yearly Progress under the criteria established under the No Child Left Behind Act.

10. More Information Available  Demographics of the sample  Within-school comparison information  Statistical information including mean differences and effect sizes For more information on this report, please contact  George Kimball, Oklahoma City Public Schools (Report will be available online at www.okaplus.ucok.edu)www.okaplus.ucok.edu For more information on Oklahoma A+ Schools, please contact  Jean Hendrickson, Oklahoma A+ Schools jhendrickson5@ucok.edu phone 405.974.3787

11. Analysis Conducted By:  George H. Kimball, Ph.D.  Director  OKLAHOMA CITY PUBLIC SCHOOLS  PLANNING, RESEARCH, and EVALUATION  413 N.W. 12th St., Oklahoma City, OK 73103  email:ghkimball@okcps.org  Phone: (405) 297-6811  Presented to the Oklahoma City Public School Board on June 19, 2006