Download presentation
Presentation is loading. Please wait.
Published byDominic Benson Modified over 9 years ago
1
PUBLIC SCHOOLS OF NORTH CAROLINA STATE BOARD OF EDUCATION DEPARTMENT OF PUBLIC INSTRUCTION 1 Review of the ABCs Standards SBE Issues Session March 2, 2005
2
PUBLIC SCHOOLS OF NORTH CAROLINA STATE BOARD OF EDUCATION DEPARTMENT OF PUBLIC INSTRUCTION 2 ABCs Accountability Basic skills with high education standards Control (at the local level)
3
PUBLIC SCHOOLS OF NORTH CAROLINA STATE BOARD OF EDUCATION DEPARTMENT OF PUBLIC INSTRUCTION 3 ABCs Accountability 1996-97 first year for grades K-8 1997-98 first year for high school
4
PUBLIC SCHOOLS OF NORTH CAROLINA STATE BOARD OF EDUCATION DEPARTMENT OF PUBLIC INSTRUCTION 4 Why Review Now? A provision in HB 1414 requires a comprehensive review of ABCs standards This same provision requires that the modified standards be in place for the 2005-06 school year
5
PUBLIC SCHOOLS OF NORTH CAROLINA STATE BOARD OF EDUCATION DEPARTMENT OF PUBLIC INSTRUCTION 5 Objectives of the Testing Program Consistent with 3 purposes in G.S. 115C- 174.10 Assure all HS graduates possess essential skills and knowledge Provide a means of identifying strengths and weaknesses in education process in order to improve instruction Make system accountable to the public for results
6
PUBLIC SCHOOLS OF NORTH CAROLINA STATE BOARD OF EDUCATION DEPARTMENT OF PUBLIC INSTRUCTION 6 1.Public and understandable 2.Multiple sources 3.Interpretations at student, school and LEA level 4.All students and groups 5.Explicit weighting 6.Avoids erroneous judgments CRESST Standards (Center for Research on Evaluation, Standards, and Student Testing)
7
PUBLIC SCHOOLS OF NORTH CAROLINA STATE BOARD OF EDUCATION DEPARTMENT OF PUBLIC INSTRUCTION 7 Related Issues Achievement levels Need clearer definitions Teachers still code same percent as Level I as in the past Teachers anticipate students’ successes regardless of previous achievement levels Process issues Various business process issues NC WISE will help Resolutions already in process
8
PUBLIC SCHOOLS OF NORTH CAROLINA STATE BOARD OF EDUCATION DEPARTMENT OF PUBLIC INSTRUCTION 8 ABCs Current Formulas Only at school level Statewide average scale score growth, index of true proficiency, index of regression to the mean More complex at the HS level Links multiple subjects State of the art for its time
9
PUBLIC SCHOOLS OF NORTH CAROLINA STATE BOARD OF EDUCATION DEPARTMENT OF PUBLIC INSTRUCTION 9 Recommendations Reviewed Technical Advisory Committee Standing group of recognized experts Supported proposal with suggestions for additional analyses Helped define high growth Provided input for minor adjustments
10
PUBLIC SCHOOLS OF NORTH CAROLINA STATE BOARD OF EDUCATION DEPARTMENT OF PUBLIC INSTRUCTION 10 Findings from Analyzing Results of Original Formulas Saw-toothed pattern: As a cohort moved through grades, % of schools making growth peaked and dipped Not necessarily tied to curriculum implementation
11
PUBLIC SCHOOLS OF NORTH CAROLINA STATE BOARD OF EDUCATION DEPARTMENT OF PUBLIC INSTRUCTION 11 More Findings EOC formulas: Drop some students from calculations Complex Curriculum changes can be problematic
12
PUBLIC SCHOOLS OF NORTH CAROLINA STATE BOARD OF EDUCATION DEPARTMENT OF PUBLIC INSTRUCTION 12 Fundamental Issues Change in curriculum prompts new test edition Change in one subject impacted other Change in curriculum expectations not mirrored in growth expectation Required conversions to old scale Saw-toothed effect
13
PUBLIC SCHOOLS OF NORTH CAROLINA STATE BOARD OF EDUCATION DEPARTMENT OF PUBLIC INSTRUCTION 13 Proposed Solutions Issue: Saw-toothed effect Solution: Averaging test performance across years increases stability
14
PUBLIC SCHOOLS OF NORTH CAROLINA STATE BOARD OF EDUCATION DEPARTMENT OF PUBLIC INSTRUCTION 14 How It Works Look at two previous years when possible Average, to find the “typical” performance of the student Use 3 years of actual student data for decisions
15
PUBLIC SCHOOLS OF NORTH CAROLINA STATE BOARD OF EDUCATION DEPARTMENT OF PUBLIC INSTRUCTION 15 Proposed Solutions Issue: Effects of curricula revisions Tie to curriculum (The relative difficulty of the grade level curricula may have shifted) Current method relies on equating study to “link” to old scale Solution: Use a scale that adjusts to difficulty of curriculum (base expectation on actual curriculum)
16
PUBLIC SCHOOLS OF NORTH CAROLINA STATE BOARD OF EDUCATION DEPARTMENT OF PUBLIC INSTRUCTION 16 How It Works Use students’ actual performance as the basis of the Change Scale (in the standard-setting year) Convert all scores to the Change Scale Calculate expectations using the Change Scale
17
PUBLIC SCHOOLS OF NORTH CAROLINA STATE BOARD OF EDUCATION DEPARTMENT OF PUBLIC INSTRUCTION 17 Proposed Solutions Issue: EOC Formula issues EOC sequence not coupled in the same manner as EOGs Solution: Simplify the formulas Fewer predictors – keep more students in the analyses Use of Change Scale allows smoother transitions between test editions
18
PUBLIC SCHOOLS OF NORTH CAROLINA STATE BOARD OF EDUCATION DEPARTMENT OF PUBLIC INSTRUCTION 18 How It Works Find the previous test(s) with the highest correlations to the current year’s tests Examples: Algebra I based on 8 th grade math Biology based on English I and 8 th grade reading Physics based on Chemistry and Geometry
19
PUBLIC SCHOOLS OF NORTH CAROLINA STATE BOARD OF EDUCATION DEPARTMENT OF PUBLIC INSTRUCTION 19 Proposed Solutions Issue: Changes in curricula and test editions Solution: Uncouple reading and math Predict reading using only reading Correlations are similar
20
PUBLIC SCHOOLS OF NORTH CAROLINA STATE BOARD OF EDUCATION DEPARTMENT OF PUBLIC INSTRUCTION 20 How It Works Original formulas used average of math and reading to provide additional information for prediction Instead, use 2 years of reading for reading prediction Correlations are better Outliers have less effect
21
PUBLIC SCHOOLS OF NORTH CAROLINA STATE BOARD OF EDUCATION DEPARTMENT OF PUBLIC INSTRUCTION 21 Proposed Solutions Issue: Conversions to old scales Multiple levels of issues Approaching third edition of math tests Solution: Use a scale-neutral system Allows true spanning of test editions No need to reset growth standards
22
PUBLIC SCHOOLS OF NORTH CAROLINA STATE BOARD OF EDUCATION DEPARTMENT OF PUBLIC INSTRUCTION 22 How It Works Based on the standard setting year of the new test edition Use the actual student performance on the test; convert to the Change Scale Use the actual state standard deviation to form the points on the Change Scale Maintain these conversion tables until curriculum and test edition change
23
PUBLIC SCHOOLS OF NORTH CAROLINA STATE BOARD OF EDUCATION DEPARTMENT OF PUBLIC INSTRUCTION 23 The Proposed Formulas Based on groups of students spanning 8 years of ABCs data Includes over 500,000 students per grade and subject (grades 3-8) Use a consistent scale Simplified to allow easier understanding Reviewed by outside experts
24
PUBLIC SCHOOLS OF NORTH CAROLINA STATE BOARD OF EDUCATION DEPARTMENT OF PUBLIC INSTRUCTION 24 AC = CS – (0.9 ATPA) Academic Change = the difference between the student’s actual performance on the curriculum-based test for this grade and his/her typical performance CS = current year score ATPA = average of two previous assessments
25
PUBLIC SCHOOLS OF NORTH CAROLINA STATE BOARD OF EDUCATION DEPARTMENT OF PUBLIC INSTRUCTION 25 AC = CS – (0.9 ATPA) The 0.9 accounts for regression to the mean All analyses point to this formula being fair Works for all grades and subjects Use 0.8 when only one previous score is available (there is more regression to the mean when using one previous test rather than the average of two)
26
PUBLIC SCHOOLS OF NORTH CAROLINA STATE BOARD OF EDUCATION DEPARTMENT OF PUBLIC INSTRUCTION 26 Anticipated Impact Results are different, especially in years with a test edition change Different formulas, different standards, different results
27
PUBLIC SCHOOLS OF NORTH CAROLINA STATE BOARD OF EDUCATION DEPARTMENT OF PUBLIC INSTRUCTION 27 How It Works – Expected Growth Calculate student level AC per subject and grade Average AC across grades and subjects within a school Grades 3-8: positive or “0” meets expected growth HS: include weighted dropout change, competency pass rate change and CTP/CUP change Consistent with original model Positive or “0” meets expected growth
28
PUBLIC SCHOOLS OF NORTH CAROLINA STATE BOARD OF EDUCATION DEPARTMENT OF PUBLIC INSTRUCTION 28 How It Works – High Growth TAC recommended use of a ratio Ratio of students with positive AC to those with negative AC (called c-ratio) HS: include dropouts in denominator; CTP/CUP graduates and competency pass rate change in numerator 1.50 or higher met high growth (if met expected growth)
29
PUBLIC SCHOOLS OF NORTH CAROLINA STATE BOARD OF EDUCATION DEPARTMENT OF PUBLIC INSTRUCTION 29 Advantages More stability Fewer fluctuations between years Clear indication of change in curriculum implementation Adjusts expectations to the curriculum Simpler formulas
30
PUBLIC SCHOOLS OF NORTH CAROLINA STATE BOARD OF EDUCATION DEPARTMENT OF PUBLIC INSTRUCTION 30 Advantages: What the Technical Experts Said More resilient due to two years’ prior performance Different results in test edition change years than previous Equitable Regression coefficients appropriate for the test relationships
31
PUBLIC SCHOOLS OF NORTH CAROLINA STATE BOARD OF EDUCATION DEPARTMENT OF PUBLIC INSTRUCTION 31 Advantages: Student Information Provides information on student performance C-ratio allows insight into how well schools or grades are implementing curriculum
32
PUBLIC SCHOOLS OF NORTH CAROLINA STATE BOARD OF EDUCATION DEPARTMENT OF PUBLIC INSTRUCTION 32 Relationship to (CRESST) Standards 1.Public and understandable 2.Multiple sources 3.Interpretations at student, school and LEA level 4.All students and groups 5.Explicit weighting 6.Avoids erroneous judgments
33
PUBLIC SCHOOLS OF NORTH CAROLINA STATE BOARD OF EDUCATION DEPARTMENT OF PUBLIC INSTRUCTION 33 Review of the ABCs Standards SBE Issues Session March 2, 2005
Similar presentations
© 2024 SlidePlayer.com. Inc.
All rights reserved.