1 Oregon Standards Evaluation Project, Contract Amendment Phase: Summary of Preliminary Findings Dr. Stanley Rabinowitz WestEd December 6, 2007

2 Contract Amendment Scope of Work Three key tasks: 1.Analysis of the degree to which Oregon’s existing content standards support the Essential Skills* 2.Analysis of the degree to which Oregon’s current test item pool (TESA/OAKS), work samples, and writing assessments assess proficiency in the Essential Skills* 3.Identification of Core Standards and recommendations for addressing gaps in coverage of core concepts* *Content Areas: ELA, math, science, and social sciences

3 Analyses are grounded in and informed by Findings from WestEd’s evaluation of Oregon’s standards and assessments Responses to Oregon’s Critical Questions Review of research literature Effective practices in other states WestEd staff expertise in the content areas and in standards and assessment development, curriculum and instruction, measurement, and cognitive development Input solicited from state stakeholder groups –Department of Education staff –Essential Skills Task Force

4 Task 1: Evidence of Support for Essential Skills Among Existing Standards Based on draft 3.0 Essential Skills (ES) Examined CIM standards in ELA, mathematics, science, and social sciences “Backward mapped” support from CIM through benchmark grades 3, 5, and 8 Presented preliminary findings to ES Task Force to inform ongoing work related to refinement of Essential Skills

5 Draft 3.0 Essential Skills 1. Read and comprehend a variety of texts at different levels of difficulty. 2. Write clearly and accurately for a variety of purposes. 3. Listen actively, speak clearly, and present publicly. 4. Think critically. 5. Apply math to solve problems in a variety of settings. 6. Use technology to learn, live, and work. 7. Demonstrate civic and community engagement. 8. Demonstrate global literacy.

6 Task 1 Findings What percentage of the CIM content standards in each of the four content areas supports the Essential Skills? Number of CIM Standards Analyzed Number of Standards Supporting an ES* Percentage of Standards Supporting an ES ELA % Mathematics84 100% Science18 100% Social Sciences393795% TotalN = % *Based on primary relationships (i.e., each standard’s support was recorded for primary link only).

7 Task 1 Findings, continued Overall, how is this support (N = 243) distributed across the Essential Skills? n = number of CIM standards, across four content areas, based on primary relationships ES 1ES 2ES 3ES 4ES 5ES 6ES 7ES 8 n=39n=31n=19n=49n=79n=2n=10n=14 16%13%8%20%32%1%4%6%

8 Task 1 Findings, continued Within each content area, how do the CIM standards support the Essential Skills?* ES 1ES 2ES 3ES 4ES 5ES 6ES 7ES 8 ELA37%30%18% 12%02%00 Math0006%94%000 Science000100%0000 Social Sciences 00033%0026%36% *Based on primary relationships.

9 Task 1 Findings, continued Of the CIM standards that support an Essential Skill, what percentage provides strong support? Number of CIM Standards Supporting ES* Number of CIM Standards Providing Strong Support Percentage of CIM Standards Providing Strong Support ELA % Math847893% Science181794% Social Sciences373081% Total % *Based on primary relationships.

10 Task 1 Findings, continued Of the CIM standards that support an ES, what percentage shows a coherent progression across grades 3, 5, 8, and CIM? Number of CIM Standards Supporting ES* Number of Standards Showing Coherent Progression Percentage of Standards Showing Coherent Progression ELA % Math847589% Science181689% Social Sciences % Total % *Based on primary relationships.

11 Summary of Task 1 Findings The distribution of standards’ primary support across Essential Skills varies by content area. –67% of ELA standards provide primary support for ES 1 and 2. –94% of math standards provide primary support for ES 5. –100% of science standards provide primary support for ES 4. –Primary support in the social sciences standards is balanced across ES 4, 7, and 8. Overall, only 2% of all standards’ primary support is associated with ES 6. In all four content areas, the majority of CIM standards (87% overall) that support an ES show a coherent progression across grades.

12 Task 2: Degree to Which State Test Items Assess Proficiency in the Essential Skills Based on draft 3.0 ES and multiple choice items in existing state pool (OAKS) –Analyses of work samples and writing assessments will be added in final report Content areas and grades included in analyses: –ELA and mathematics / grades 3, 5, 8, CIM –Science and social sciences / grades 5, 8, and CIM

13 Task 2 Findings Overall, what percentage of items assesses proficiency in an Essential Skill? Number of Items Analyzed Number of Items Assessing an ES Percentage of Items Assessing an ES ELA2, % Math1, % Science1, % Social Sciences % TotalN = 5,6644,912 87%

14 Task 2 Findings, continued How are these items (N = 4912) distributed across the eight Essential Skills? n = number of items, across the four content areas *Writing and listening are not assessed via OAKS multiple choice items. ES 1ES 2*ES 3*ES 4ES 5ES 6ES 7ES 8 n = 2,253 n = 0 n = 406 n = 1,566 n = 6 n = 464 n = %8%32%<1%9%4%

15 Task 2 Findings, continued At each grade, of those items that assess an Essential Skill, what percentage of items assesses each ES? ES 1ES 2ES 3ES 4ES 5ES 6ES 7ES 8 Grade 347%00053%000 Grade 547%006%30%<1%13%3% Grade 849%006%26%<1%12%7% CIM42%0017%26%<1%9%6%

16 Task 2 Findings, continued In each content area, of those items that assess an Essential Skill, what percentage of items assesses each ES? ES 1ES 2ES 3ES 4ES 5ES 6ES 7ES 8 ELA100% Math %000 Science2%0096%2%1%00 Social Sciences 5% %30%

17 Task 2 Findings, continued Of those items that assess an Essential Skill, what percentage demonstrates a strong linkage to that Essential Skill? Number of Items Assessing an ES Number of Items with Strong Linkage to that ES Percentage of Items with Strong Linkage to that ES ELA2,2092,19399% Math1,5581,54199% Science % Social Sciences % TotalN = 4,9124,55493%

18 Next Task Step 3: Identification of Core Standards –Which existing content standards meet the criteria for “core standards”? –Which core concepts are not embodied in the existing standards (“gap” analyses)? ELA grades K-8 and CIM Math CIM only (state has completed K-8) Science grades 3, 5, 8, and CIM Social sciences grades 3, 5, 8, and CIM

19 For more information about the standards and assessment evaluation: standards/Standards_Review.aspx For more information about WestEd: