Goals for Webinar: Applications of SEC Alignment Analysis What is SEC alignment method How alignment analyzed and reported --Stands/Assess: 30 states, NAEP, International; next: Common Core How states, districts, schools use results

Purposes: Alignment Analysis Policy question: Does Assessment match standards? Curriculum to standards, Standards to standards? Local question: Does instruction meet intended Curriculum or Goals? Question of How? What Measures?

Vertical and Horizontal Alignment Alignment= Core Idea in standards-based education reform (Porter, 2002) Vertical and Horizontal Alignment

State-focused Models Webb: Assessment to standards Content consistency, Depth of know, Balance External teams, Statistics, Reports Achieve: Standards, Assess to Standards Content centrality, Expected Perform, Challenge Ext. teams, Stats, Qualitative, Reports SEC: Standards, Assessment, Instruction Content matrix and language --multiple comparisons/levels; Cross-state teams Graphical online reporting (maps, tiles)

Surveys of Enacted Curriculum The intended curriculum: State content standards—What students should learn The assessed curriculum: State (and other) assessments—tested learning A neutral content grid with cognitive demand The SEC instrument provides a cost/time efficient way to compare state or district standards to state or district assessments, the curriculum, the “enacted” curriculum (what teachers teach), and the learned curriculum (student achievement). Content is examined two-dimensionaly–content topics as well as level of cognitive demand (expectations teachers have for the kind/level of learning required of students to master/understand that content topic). The learned curriculum: Student outcomes based on school learning The enacted curriculum: What teachers teach

www.SECsurvey.org www.Seconline.org Teachers go to this website to take the surveys

Content Matrix--Math

Surveys of Enacted Curriculum Surveys of Enacted Curriculum (SEC) are a research-based data tool that allows teachers, administrators and policy makers to examine the extent of alignment between the enacted curriculum (what teachers teach), the intended curriculum (what the standards require), and the assessed curriculum (what the state tests). Council of Chief State School Officers (CCSSO) www.seconline.org ©2007. All rights reserved. 9

The SEC are….. Surveys with teachers by K-12 subject: Math, Science, ELA, SST, and now ELD teachers, to collect and report consistent data on current instructional practices and content being taught in classrooms. Content analysis of Stands., Assessments through SEC Collaborative –CCSSO and WCER Council of Chief State School Officers (CCSSO) www.seconline.org ©2007. All rights reserved. 10

Background State curriculum specialists, teachers, and researchers developed the SEC and reporting tools over the past decade. The Surveys have been field tested in hundreds of schools and classrooms through collaboration with state education agencies. The SEC are now being used in state and local projects across the U.S. The data collection, analysis, and application services are available through CCSSO and several collaborating partner organizations. Council of Chief State School Officers (CCSSO) www.seconline.org ©2007. All rights reserved. 11

Research Says…. Reform-Up-Close (Porter, Kirst, Osthoff, Smithson, Schneider, 1993) Validation of teacher self-report survey data. Upgrading Mathematics (Gamoran, Porter, Smithson, White, 1997) Predictive validity of alignment index comparing instruction & assessments Data on Enacted Curriculum (Blank, Porter, Smithson, 2004) Use of SEC data to facilitate school improvement efforts Some question how valid and reliable the SEC are due to teachers providing their perception. Curriculum mapping is also teacher perception- are we questioning the validity and reliability of these? Here are examples of the research foundation upon which the Surveys of Enacted Curriculum are based. MSP-PD Study (Blank, Smithson, Porter, Garet, Birman, 2005) Use of SEC data for program evaluation  SEC – ELL Consortium (2008-10) – Analyzing alignment of content AND language of academic content Council of Chief State School Officers (CCSSO) www.seconline.org ©2007. All rights reserved. 12

Webb’s “Depth-of-Knowledge” Model for Alignment Describe and analyze the rigor, or complexity, of content and expectations between standards/framework and assessments. Level 1: Recall and Reproduction Level 2: Skills and Concepts Level 3: Strategic Thinking Level 4: Extended Thinking

SEC Home Page This is the home page of the SEC online website. Not only is this where the teachers register to take the survey, but also where you can find resources and access your data.

Topics List When teachers take the survey - they are asked about content in a two dimensional way- Topics…… Council of Chief State School Officers (CCSSO) www.seconline.org ©2007. All rights reserved. 15

SEC Math Expectations for Students Here is an example of the cognitive demand categories for mathematics. The SEC is asking teachers to reflect back on the instruction they provided and and think beyond the list of topics into asking them to recognize the level of expectation or rigor as well. Council of Chief State School Officers (CCSSO) www.seconline.org ©2007. All rights reserved. 16

Important SEC facts Confidential Teachers create unique usernames and passwords when they register The data reports out in groups of 3 or more teachers Completion of the survey is the beginning of your SEC experience Here are some important facts to keep in mind. As Districts and schools are going through school Improvement processes, they are often asked to fill out a plan or template that requires more evidence than ever before in identifying needs. Some responses to this are to create paper surveys on climate, PD, alignment, etc. Then someone has to collect the surveys and summarize the information. The SEC allows for teachers to accomplish all these different surveys as well as allowing teachers to be truly honest about their responses. The data is readily available once the surveys are completed online. Once complete the survey data allows for in depth conversations about instruction and practice based on the data the teachers provided.

Sophistication of knowledge Students use the same basic skills but in a more sophisticated way. Same rules, higher level game…. And what we call Cognitive demand or expectations for students. Council of Chief State School Officers (CCSSO) www.seconline.org ©2007. All rights reserved. 18

Instructional Practices Use of Homework Instructional Activities Small Group Work Educational Technology Teacher Opinions and Beliefs Professional Development Instructional Influences Instructional Readiness The SEC also asks teachers about their practices, strategies and beliefs

Report Generator Once teachers complete the survey, they can then go into SEC reports and view their data or leadership can view groups of teacher data. The report generator allows for teachers and administrators to compare of different sets of data.

Teachers complete the Survey questions through an online, web-based system. Upon completion, the group data are reported in user friendly charts and graphs to facilitate analysis of the results. Council of Chief State School Officers (CCSSO) www.seconline.org ©2007. All rights reserved. 21

The SEC data Assist in ･Aligning classroom instruction with state standards and assessments ･Evaluating effects of initiatives, such as professional development, in changing instructional content and/or practice ･Analyzing instructional practices and teacher preparation to develop a needs assessment in low performance areas ･Measuring indicators of instruction and their relationship to student achievement Council of Chief State School Officers (CCSSO) www.seconline.org ©2007. All rights reserved. 22

Enacted (3rd Grade Teacher Reports) Intended (3rd Grade Math Indicators) Here is an example ofthe resulting data. This is a contour map comparing teachers on the left side to standards on the right . The lighter the color the less emphasis or time, the darker the color, the more emphasis or time. This is a coarse grain chart. By clicking on any of the text to the right of the map you can then access a fine grain char.t

Enacted (3rd Grade Teacher Reports) Intended (3rd Grade Math Indicators) Fine grain look at “data analysis”

Intended (3-4 Math Benchmarks) Assessed (3rd Grade Math Achievement Test) 3-4 Math Benchmarks vs. 2005 3rd grade math achievement test.

Enacted (9th Grade Teacher Reports) Intended (9th Grade Indicators) Another example, this one ELA, 9th grade teachers vs 9th grade indicators We could look at tests, vs. standards, building level, grade level, etc.

Reading First: 3rd Grade Teachers Random Sample: 3rd Grade Teachers This slide compares the teacher reports of the Enacted curriculum between Reading First 3rd grade teachers and the Random Sample 3rd grade teachers. There is very high alignment.

Tile Chart Preview- grab white

The SEC has an entire section in the Instructional practices Part of the Survey focusing on Professional Development. In this example, the questions compare time spend on PD in workshops and in-service, summer institutes and conferences, between the Reading First and the Random Group. In this example, the RF group reported spending more time in workshops and conferences than the Random Group. Additionally, Grade 1 RF teachers reported slightly higher numbers than both grade 2 and Grade 3 (Grade 4 is shown on this chart as a default function of the report generator, but the data is not relevant). Teachers in both groups spent about the same amount of time in summer institutes or conferences

SEC Project Planning Guide The SEC Planning Guide is a tool developed to assist regional, district and school leaders. The planning guide is way to organize your SEC project. It will be used throughout the SEC 101 workshop. Although the guide used in the SEC 101 will be tentative, it is a way to engage those interested in thinking about resources and ways that the SEC will assist them. It is intended to get at questions that usually come up once planning has begun back in their region or district. At this time take the list of curriculum work that you generated earlier and list those under step one- realizing that is VERY tentative.

Where are the resources to assist you? www.seconline.org www.secsurvey.org Council of Chief State School Officers (CCSSO) www.seconline.org ©2007. All rights reserved. 31

Education Week, October 8, 2003 “Regardless of what a state policy requires or what a district curriculum spells out,” says Andrew C. Porter of Vanderbilt University, “the classroom teacher ultimately decides” what to teach. Education Week, October 8, 2003 Source: Maine SEC project PPT, 2004