1 Oregon Content Standards Evaluation Project, Contract Amendment Phase: Preliminary Findings Dr. Stanley Rabinowitz WestEd November 6, 2007

2 Background: Overview of Standards Evaluation Project One of several key projects funded by the Gates Foundation Independent evaluation by WestEd, a non-profit educational research, development, and service agency

3 Oregon Content Standards and Assessment System Evaluation WestEd was contracted to Review and evaluate the content standards (all grades, all academic content areas) Evaluate alignment among the state assessments and the content standards Make recommendations for improvement of Oregon’s comprehensive standards and assessment system

4 Contract Amendment Phase Extended scope of work to assist the ODE in implementing key recommendations that emerged from the comprehensive evaluation Builds upon current state initiatives Time frame: work will extend from mid- October to December 31, 2007

5 Contract Amendment Scope of Work Three key steps: 1.Analysis of the degree to which Oregon’s existing content standards support the draft Essential Skills 2.Analysis of the degree to which Oregon’s current test item pool (TESA/OAKS), work samples, and writing assessments assess proficiency in the Essential Skills 3.Identification of Core Standards and recommendations for addressing gaps in coverage of core concepts

6 Focus for Today: Step 1 Support for Essential Skills in Existing Standards Based on draft 3.0 Essential Skills (ES) Evidence of degree to which current content standards in ELA, mathematics, science, and social sciences support the ES Backward mapping of support from CIM through benchmark grades 3, 5, and 8 Recommendations for improving/refining draft Essential Skills to ensure effective application across content areas

7 Step 1 analyses are grounded in and informed by Findings from the evaluation of Oregon’s standards and assessments Responses to Oregon’s Critical Questions Review of research literature Trends in practices across other states WestEd staff expertise in the content areas and in standards and assessment development, curriculum and instruction, measurement, and cognitive development Input solicited from state stakeholder groups –Department of Education staff –Essential Skills Task Force

8 Draft 3.0 Essential Skills 1. Read and comprehend a variety of texts at different levels of difficulty. 2. Write clearly and accurately for a variety of purposes. 3. Listen actively, speak clearly, and present publicly. 4. Think critically. 5. Apply math to solve problems in a variety of settings. 6. Use technology to learn, live, and work. 7. Demonstrate civic and community engagement. 8. Demonstrate global literacy.

9 Protocol for Step 1 Determine whether each CIM standard supports a draft Essential Skill 1.Which one (if any) of the eight ES does it primarily support? 2.What is the strength of that relationship? Strong- CIM standard supports a fundamental skill or concept as explicitly stated in the Essential Skill Partial- CIM standard supports the Essential Skill in a superficial way or at a lower complexity level Backward map from the CIM standard to identify evidence of support for the ES in grades 3, 5, and 8 –Do standards in grades 3, 5, 8, and CIM support the Essential Skill in a logical and coherent sequence or progression?

10 Guidelines Used During Analyses Analysts focused on the bold category- level statements that defined each Essential Skill. Where provided, the numbered indicator statements were used to clarify the focus of each Essential Skill. A CIM standard could support an Essential Skill without specifically addressing one of the indicator statements provided.

11 Preliminary Findings What percentage of the CIM content standards in four content areas support the Essential Skills? Number of CIM Standards Analyzed Number of Standards Supporting an ES* Percentage of Standards Supporting an ES ELA % Mathematics84 100% Science18 100% Social Sciences393487% totalN = % *Based on primary relationships (i.e., each standard’s support was recorded for primary link only).

12 More Preliminary Findings… Overall, how was this support distributed across Essential Skills? n = number of CIM standards, across four content areas, based on primary relationships ES 1ES 2ES 3ES 4ES 5ES 6ES 7ES 8 n=46n=32n=21n=34n=84n=1n=11n=12 19%13%9%14%34%<1%5%

13 More Preliminary Findings… Within each content area, how do the CIM standards support the Essential Skills?* ES 1ES 2ES 3ES 4ES 5ES 6ES 7ES 8 ELA (all standards support an ES) 44%30%20% 5%01%00 Math (all standards support an ES) %000 Science (all standards support an ES) %0000 Social Sciences (34/39 standards support an ES) 00028%00 31% *Based on primary relationships.

14 More Preliminary Findings… Of those CIM standards that support the Essential Skills, what percentage was rated as providing strong support? Number of CIM Standards Supporting ES* Number of Standards Providing Strong Support Percentage of Standards Providing Strong Support ELA % Math847893% Science181689% Social Sciences342368% total % *Based on primary relationships.

15 More Preliminary Findings… What percentage of CIM standards that support the ES show a coherent progression across grades 3, 5, 8, and CIM? Number of CIM Standards Supporting ES* Number of Standards Showing Coherent Progression Percentage of Standards Showing Coherent Progression ELA % Math847286% Science181689% Social Sciences % total % *Based on primary relationships.

16 Summary of Preliminary Findings Overall, 98% of existing standards across the four content areas (ELA, math, science, and social sciences) primarily supported one of the draft Essential Skills. Across content areas, the majority of relationships among supporting standards and the Essential Skills was strong. –ELA and math: >92% support was strong –Science: 89% support was strong –Social sciences: 68% support was strong

17 Summary of Preliminary Findings The distribution of standards’ primary support across Essential Skills varied by content area. –Overall, less than 10% of standards’ primary support was associated with ES 6, 7, and 8 combined. –Primary support in the ELA standards was balanced across three ES (1, 2, 3). –100% of math standards provided primary support for ES 5. –100% of science standards provided primary support for ES 4. –Primary support in the social sciences standards was balanced across three ES (4, 7, and 8). Across content areas, evidence of coherent progression across grades ranged from 83-91% (85% overall).

18 Next Steps Analysis of the degree to which Oregon’s current test item pool (TESA/OAKS), work samples, and writing assessments assess proficiency in the Essential Skills Identification of Core Standards and recommendations for addressing gaps in coverage of core concepts Formal report with findings and recommendations for improvement/ refinement emerging from the Amendment Phase

19 For more information about the standards and assessment evaluation: andards/Standards_Review.aspx For more information about WestEd: