1. Copyright © 2008 Learning Point Associates. All rights reserved. Surveys of the Enacted Curriculum Cary Cuiccio Lisa Palacios March 6, 2009

2. www.learningpt.org 2 Learning Point Associates Our Vision An education system that works for all learners. Our Mission We deliver the knowledge, strategies, and results to help educators make research-based decisions that produce sustained school improvement.

3. www.learningpt.org Research has found that faculty in successful schools always question existing instructional practice and do not blame lack of student achievement on external causes. — Carl Glickman, 2002

4. www.learningpt.org What if? … there was a powerful tool that compared teacher’s reflections of content instruction to state standards, benchmarks and assessments? …you could compare how your teachers teach content compared to how others teach across the nation? …you could have consistency across grade levels on both time teachers spend on content as well as depth of instruction?

5. www.learningpt.org What if? … there was a tool that compared teacher’s reflected instructional practices to other teachers’ across a school, district or the nation? …you could use data on instructional quality and content to guide professional development? …you could use anonymous teacher data to start a powerful school discussion about teachers’ curricular and professional development needs based on data and observed differences between student achievement scores and schools curriculum?

6. www.learningpt.org Introducing: The SEC  Online web-based surveys in Mathematics, English Language Arts, Social Studies and Science.  Groups data that are reported in user-friendly charts.  Assists in facilitating group discussions for improved instruction across classrooms, schools and districts.  A practical, reliable set of data collection tools used by teachers.

7. www.learningpt.org 7

8. State Departments of Education (selected) National Organizations (College Board, ACT, NAEP, WIDA) Council of Chief State School Officers Wisconsin Center for Education Research Learning Point Associates TERC SEC Collaborative

9. www.learningpt.org History of the SEC  The Surveys of Enacted Curriculum (SEC) Collaborative Project was initiated in 1998.  Surveys are available K-12 for Mathematics, Science, English Language Arts and Reading, and Social Studies.

10. www.learningpt.org History of SEC (cont.)  Over the past ten years the Council of Chief State School Officers (CCSSO) and their partners have worked with states to implement the SEC tools for data collection, analysis, and reporting.

11. www.learningpt.org Research on the SEC MSP-PD Study (Blank, Smithson, Porter, Nunnaley, Osthoff, 2006) Use of SEC data for program evaluation Reform-Up-Close (Porter, Kirst, Osthoff, Smithson, Schneider, 1993) Validation of teacher self-report survey data. Upgrading Mathematics (Gamoran, Porter, Smithson, White, 1997) First content analysis of assessment using content language. Predictive validity of alignment index comparing instruction & assessments Data on Enacted Curriculum (Blank, Porter, Smithson, 2004) Use of SEC data to facilitate school improvement efforts First content analysis of state standards

12. Copyright © 2008 Learning Point Associates. All rights reserved. The ELA Survey Part One: Instructional Content and Cognitive Demand

13. www.learningpt.org Instructional Content A tool to address a key question:  Is classroom instruction aligned with state content standards and assessments? Content Emphasis Cognitive Expectations

14. www.learningpt.org Instructional Topics  Topics provide the framework for data collection and comparison.  Topics include the range of topics found in English Language Arts classrooms, K–12.  Stable framework—not intended for a particular state or curriculum; rather with the content area in mind generally.  Neutral matrix—intends to include all possible topics that could be taught.

15. www.learningpt.org

16. 100Phonemic awareness1000 Elements of Presentation (Verbal and Written) 200Phonics1100Writing applications 300Vocabulary1200Language Study 400 Text and print features 1300Listening and Viewing 500Fluency1400Speaking and Presenting 600Comprehension1500Forms of Text 700Critical Reasoning1600Genre (fiction or non-fiction) 800Author's craft1700Sources of Text 900Writing processes1800Choice K-12 English Language Arts Reading Content Areas

17. www.learningpt.org

18. 600Comprehension 601Word meaning from context 602Phrase 603Sentence 604Paragraph 605Main idea(s), key concepts, and sequence(s) of events 606Descriptive elements (e.g., detail, color, condition) 607Narrative elements (e.g., events, characters, setting, and plot) 608Persuasive elements (e.g. propaganda, advertisement, and emotional appeal) 609 Expository or informational elements (e.g., explanation, lists, and organizational patterns such as description, cause-effect, and compare-contrast) 610Technical elements (e.g., bullets, instruction, form, sidebars, etc.) 611Electronic elements (e.g., hypertext links, animations) 612 Strategies (e.g., activating prior knowledge, questioning; making connections, predictions; inference, imagery, summarization, re-telling) 613Self-correction strategies (e.g., monitoring, cueing systems, and fix-up) 614Metacognitive processes (e.g., reflecting about one's thinking) 615Interpreting maps, graphs, charts 616Test-taking strategies 690Other K-12 English Language Arts Reading Specific Content

19. www.learningpt.org Cognitive Expectations  State standards and assessments have inherent cognitive expectations.  Instructional activities also have cognitive expectations.  The SEC measures the cognitive expectations of instruction.

20. www.learningpt.org Cognitive Expectations English Language Arts and Reading  Memorize and Recall  Perform Procedures and Explain  Generate, Create, and Demonstrate  Analyze and Investigate  Evaluate

21. Copyright © 2008 Learning Point Associates. All rights reserved. Data Interpretation Part One: Instructional Content/Cognitive Demand aligned to State Standards Contour Maps Tile Charts

22. www.learningpt.org Survey Instrument: Alignment  Subject Content  Student Specific Topics  Time Spent on Topic  Student Expectations

23. www.learningpt.org Reading SEC Contour Maps

24. www.learningpt.org

25. This data is from 134 Grade 3 teachers. Alignment describes how well your instruction aligns to standards or assessments. 1.0 is perfect; anything above.5 is fine. You read this like a topographical map. Darker areas are peaks and represent more time spent on that topic at that cognitive demand level. Here, the 3 rd grade teachers are spending more time on phonemic awareness and phonics, and at more levels of cognitive demand, than the standards require. Same here in terms of topographical maps. Teachers are spending less time on speaking and presenting than the standards require. This map is the NY state standards specific to your grade. You can also look at the NY state assessments. The map on this side will be your data, and only you can access it. That’s why this data is from an entire grade. You can also click on any subtopic and get another map comparing your teaching to standards and/or assessments at a “fine grain” level.

26. www.learningpt.org

37. Copyright © 2008 Learning Point Associates. All rights reserved. Activity

38. The Survey Part Two: Instructional Practices

39. www.learningpt.org Instructional Practices A tool to address key questions:  Are instructional practices optimized for student achievement?  Are the professional development efforts aligned with needs and priorities?  Are teachers’ opinions and belief in line with district and school initiatives?

40. www.learningpt.org Instructional Practices  Homework  Classroom assessments  Opinions and beliefs  Classroom Instructional preparation  Instructional influences  Professional development  Personal characteristics  Formal course preparation

41. Copyright © 2008 Learning Point Associates. All rights reserved. Taking the Survey

42. www.learningpt.org www.seconline.org

43. www.learningpt.org Completing the Online Survey  Approximately 60-90 minutes to compete.  High-speed Internet connection required.  May be completed in multiple sittings.  Data are saved as each section is submitted.

44. Copyright © 2008 Learning Point Associates. All rights reserved. Data Interpretation Part Three: Powerful Discussions

45. www.learningpt.org And so you just threw everything together? Matthews, a posse is something you have to ORGANIZE.

46. www.learningpt.org What does the SEC show? Instruction Assessment Standards Intersection of what is taught with what is tested. Intersection of what is taught with what is in standards. Intersection of what is tested with what is in the standards. Taught, tested, and in the standards.

47. www.learningpt.org Traditional Model for School Improvement Identify “Best Practices” for Achievement Problems Identify Shortcomings in Student Performance

48. www.learningpt.org Better School Improvement Identify Appropriate Programmatic Solutions Identify Shortcomings in Student Performance Review Instructional Program Data Relative to Concerns The Surveys of Enacted Curriculum

49. www.learningpt.org Applications for SEC  Aligning standards, assessments, and curriculum  Improvement of instruction  Interpreting student assessment results  Needs assessment and program evaluation  Indicators system for monitoring progress

50. www.learningpt.org Possible Data Analysis Outcomes  Targeted professional development  Changes in environmental factors  Positive climate of change  Continuous improvement process  Review of curriculum materials  Other data-driven decisions and actions

51. www.learningpt.org What SEC isn’t --  Individual Teacher Evaluation The “wall” of 3  A tool to judge alignment without school/district context and/or discussion with staff.  The only piece of data for measuring alignment of teacher instruction.

52. www.learningpt.org “Regardless of what a state policy requires or what a district curriculum spells out,” says Andrew C. Porter of Vanderbilt University, “the classroom teacher ultimately decides” what to teach. Education Week, October 8, 2003

53. www.learningpt.org Think, Pair, Share  Discuss what you’ve learned with the person next to you and make a few notes. What was the purpose today? Similarities in your thoughts, perspectives? Based on what you read – how could the SEC help to improve your classroom, school or district?

54. www.learningpt.org 54 Cary Cuiccio P: 718-483-4279 E-Mail: cary.cuiccio@learningpt.org Lisa Palacios P: 630-649-6601 E-Mail: lisa.palacios@learningpt.org