1. Aligning Instruction to Standards and Assessments for English Language Learners (Meeting #1) U.S. Department of Education, Title I, Enhanced Assessment Grant December 6-7, 2007 Iowa SEC-ELL Consortium

2. Study Objectives  Do ELLs have opportunity to learn academic content and skills in state standards?  What is relationship of state ELD standards and assessments to state academic standards?  What instructional practices/strategies are used to teach English language skills?  What is relationship of alignment of instruction (to standards) with student achievement?

3. Major Steps I. Planning & Instrument Development II. Alignment content analysis --workshop III. Data collection – in schools w/teachers IV. Produce data analysis reports -- from project V. Technical assistance, training—state, local VI. Achievement analysis, Further data (options) VII. Report and Disseminate

4. SEC-ELL Consortium Iowa, Florida, Idaho, Maine, Minnesota, Ohio, Utah, Virginia, Wisconsin, Nevada CCSSO – Rolf Blank Carlise Smith, Adam Peterman Carolyn Karatzas, Lani Seikaly, consultants edCountEllen Forte WCER – John Smithson Jaime Usma WestEd --Edynn Sato Peter Worth, Merle Longenecker EvaluatorJoe McCrary (WestEd) AdvisersJamal Abedi, Gary Cook, Robert Linquanti, Charlene Rivera, Andrew Porter, Phoebe Winter

5. Meeting #1 Objectives  Review study design and decide how project will meet state needs and education community  Develop ELL component or extension of SEC instrument for ELL alignment  Project schedule and implementation plan

6. Tools for Aligning Instruction, Standards, & Assessments Orientation Surveys of Enacted Curriculum

7. SEC Orientation—Leaders, Educators What? … are the Survey of Enacted Curriculum tools? How?… are data collected, analyzed, reported? Why?… are SEC data useful to educators, leaders, researchers? Now what?…how do we implement locally and lead the use of SEC in schools?

8. Rationale Assessment Standards Curriculum

9. Applications Alignment analysis --instruction, standards, assessments Instructional improvement in schools Needs assessment/ Evaluation Indicators – monitoring change over time

10. Key Question -- SEC Tools Research into Practice How can Educators obtain reliable, valid data to determine Alignment of instruction with required standards and assessments?

11. School & Class Description Instructional Activities General Problem Solving Activities Pairs & Small Group Work Use of Hands-on Materials Use of Calculators/Computers & other Ed. Tech. Assessment Use Instructional Influences Instructional Readiness Teacher Opinions Professional Development Types, Frequency Content, Active, Collegial, Coherence Instructional Content Topic x Cog. Demand Survey Sections

13. Content Maps

14. Steps in SEC Development R&D studies of curriculum-- Early ‘90s Models:TIMSS, NAEP, Analyze assessment OTL interest of States, Porter/ Smithson research CCSSO Science Assessment project Develop Math/Science surveys-’98-’01 (NSF) 11-State field study, Reports format, Alignment method

15. Multi-State Collaborative  Build Knowledge of leaders  Share Strategies  Produce joint products, tools

16. SEC Collaborative Members 2007-08 Delaware Idaho Illinois Indiana Iowa Kansas Maine Michigan Mississippi Montana Ohio Oklahoma Oregon Wisconsin Duval/Jax, FL Minnesota Vermont

17. Social Studies SEC Development 2005-2007 Arizona Delaware Idaho Maine Ohio Oregon Vermont Wisconsin Minnesota Nevada U of Wis-WCER NCSS Civics Education National Geographic GENIP NCEE CCSSO

18. SEC Collaborating Organizations Council of Chief State School Officers www.SECsurvey.org www.SECsurvey.org Wisconsin Center for Education Research www.SEConline.orgwww.SEConline.org Learning Point Associates/NCREL www.SECsupport.org www.SECsupport.org TERC Regional Alliance DEC Project www.ra.terc.edu/DEC www.ra.terc.edu/DEC Measured Progress (assessment development and PD)

19. SEC Collaborative offers Core Components of SEC tools/services 1) Surveys with teachers – report instruction 2) Alignment analysis – content code standards and assessments 3) How to use Data – knowledge, skills 4) Leader development – 2-3 meetings/yr 4) In-state workshops: a) orientation b) use of data

20. SEC by the Numbers: ’06-07 10,393 SEC Surveys 06-07 4674 Math 1993 Science 3646 ELAR 80 Soc Stud 131 Standards/Assess. Content analysis 1000 est. schools--Data use

21. SEC by the Numbers (2) By Comparison 1996 207 Science Teachers (5 states) 1999 626 M/S Teachers (11 states) 2005 5,414 E, M, S teachers 2006 10,200 E, M, S teachers 2007 10,400 E, M, S, SSt (27 states)

22. SEC Collaborative How SEC Collaborative works for members  Training and Assistance to project leaders  Project Planning w/ budget– meet local objectives  Each state/district makes decisions on services & project plan  Multi-state benefits of collaborative e.g., learning from experience, sharing ideas, training

23. Accomplishments 06-07 SEC Collaborative  In-state Leader workshops, Project Planning  PPTs for Orientation, Survey Admin, Data use  Leader Development Standards document  Website revisions, improvements (SECsurvey.org; SEConline.org)  E.g., State Projects, All Alignment results, Report generator

24. More Accomplishments SEC Collaborative  “Using Data to Improve Instruction” St. Louis 9/07 conference – 200 leaders, 32 sessions (30 states)  Social Studies Survey development –  10 states, 5 prof. orgs.,  NAEP Alignment analysis – 2007 Math items  Research/evaluation:  MSP study of professional development– multi-site study  Ohio case study of district/school implementation of SEC  Follow-up data tool –to track school level data  Validity study—longitudinal data for teachers and students  Ohio reading study  John demonstrated SEC comparisons in Feb 07

25. Goals SEC 07-08  State/local projects—From pilots to broad application, local leadership training  Integrate SEC tools/data with Improvement Initiatives, especially data-driven  Research/Evaluation --Broader use of Surveys, Alignment  Assist States in planning with Service delivery agencies;  Social Studies SEC complete & implement

26. (2) Steps in SEC  Data as PD in urban schools – DEC study  Evaluation of PD effects on instruction (MSP study, 4 sites) English/ Language Arts survey (‘03 – 04) SEC Collaborative –Operational—15 to 20 states – since ‘04 Social Studies survey Using Data leadership and PD SEC–ELL development See www.secsurvey.org / Resourceswww.secsurvey.org/

27. Questions Addressed by SEC Data How can in-depth data on content of instruction be collected and reported (not topic checklists), to analyze teaching content in relation to standards, assessments, achievement? How can methods of teaching practices be compared across classrooms, schools, districts, and states? How can enacted curriculum data be reported in a manner to encourage use by teachers to improve instruction?

28. Key Education Questions (cont’d) How can we measure the effects of standards- based initiatives on instructional practices and curriculum in classrooms? How can we analyze effectiveness of professional development on changes in teachers’ instructional practices? (i.e. determine the quality of professional development)

29. Content Matrix

30. Science content matrix

31. Surveys of Enacted Curriculum A neutral content grid with cognitive demand The intended curriculum: State content standards— What students should learn The enacted curriculum: What teachers teach The learned curriculum: Student outcomes based on school learning The assessed curriculum: State (and other) assessments— tested learning

32. The Enacted Curriculum

35. SEC Websites CCSSO, SEC Collaborative www.SECsurvey.org WCER, SEC Online Survey and Reports www.SEConline.org