1. Using Curricular Measures for Description & Analysis SEC Collaborative Meeting Tampa, FL February 21, 2005 John L. Smithson, Director, Measures of Enacted Curriculum Wisconsin Center for Education Research, University of Wisconsin-Madison johns@education.wisc.edu Surveys of Enacted Curriculum

2. The SEC Data - Sets Distinctions On-Line Off-line Data collection/processing/reporting Descriptive Data Limited Reporting Options Easy Access / Indiv. Results Analytic analyses Unlimited reporting options Requires data manipulation

3. Conducting Inquiry Using SEC Data Forms of Inquiry CollaborativeorEvaluative Teacher Enrichment School Improvement Professional Lrng. Comm. Program Evaluation Indicator Reporting Program Management Performance Modeling

4. MSP-PD Study (In progress: Blank, Smithson, Porter, Garet, Birman) Use of SEC data for program evaluation A short history of SEC research Reform-Up-Close (Porter, Kirst, Osthoff, Smithson, Schneider, 1993) Validation of teacher self-report survey data. Upgrading Mathematics (Gamoran, Porter, Smithson, White, 1997) First content analysis of assessment using content language. Predictive validity of alignment index comparing instruction & assessments Data on Enacted Curriculum (Blank, Porter, Smithson, 2004) Use of SEC data to facilitate school improvement efforts First content analysis of state standards

5. Alignment Relationships in Standards-based Reform

6. Instruction Assessment Standards Intersection of what is taught with what is tested. Intersection of what is taught with what is in standards. Intersection of what is taught with what is in the standards Taught, tested, and in the standards Alignment Relationships in Standards-based Reform

7. A Quantitative Approach to Alignment Porter-Smithson/SEC Alignment Process Content analyses of curriculum documents and reports of practice by content experts using two-dimensional content language. Multiple raters (w/ content & assessment expertise) using independent ratings in combination with team discussions. Content Description [Topic(s) by Cognitive Demand(s)] Yields Alignment Index based on:

8. Alignment Analyses The analytic power of quantitative alignment measures Prediction Program Evaluation* Planning Performance Modeling * Not teacher evaluation

9. Alignment Analyses Students perform best when tested on material for which they have been provided the opportunity to learn. Class GainsStudent Gains Alignment Index0.4510.259 Correlation of Alignment Index to Achievement Gains From: Gamoran, et.al. (1997). Uprgading High School Mathematics Instruction. EEPA v19n4pp325-338.

10. Explaining variation in student learning gains

11. Alignment Index: Instruction to Standards Mathematics Across 4 Districts Alignment Analyses for School Improvement Using alignment as an outcome measure (Measuring change in alignment over time) Treatment 99 Control124 Leaders 16 Counts

12. Alignment Analyses for Planning Purposes

13. Mapping Curriculum Materials

14. (www.seconline.org) SEC - Online

15. Administrative Functions: Administration Set-up Review Registrants, Completion Rates Administrator Report Generator

17. SEC Reports Floating Bars Dynamic Sample Selection Dynamic Data Disaggregation Sample and Disaggregate Counts Comparison Charts Individual Results

18. SEC Reports: Instructional Content Coarse Grain Maps Content Areas by Cognitive Demand

19. SEC Reports: Instructional Content Fine Grain Maps Topics by Cognitive Demand

20. Alignment as a Systemic Tool Fine Grain

21. Calculating Alignment 1-((  |x-y| (1-n) )/2)