Summer Leadership Institute New Test Item Specifications for High School Courses: Implications for Upcoming Common Assessments and Opportunities for Involvement Heather P. Wright August 9-10, 2012

Common Board Configuration Date: August 9 & 10, 2012 Vocabulary: Item Specifications Bell Ringer: How do your teachers use item specifications? Agenda: Gradual Release Learning Goal:Understanding the multiple implications of new test item specifications for previously untested courses Benchmark: N/A Objective: Participants will understand the creation process for the new test item specification for high school “gap” courses, will understand the instructional implications of these item specifications, and will learn about upcoming opportunities for involvement in the “Gap” course assessment project. Summarizing Activity: Consider how you will use test item specifications in your school to improve instruction. Essential Question: What is the creation process for the test item specifications for high school “gap” courses? How do these test item specifications affect upcoming common assessments? What professional development opportunities and opportunities for involvement are available for Lake County teachers? Homework: Encourage your teachers to participate as item writers and reviewers.

Lake County Schools Vision Statement A dynamic, progressive and collaborative learning community embracing change and diversity where every student will graduate with the skills needed to succeed in postsecondary education and the workplace.   Mission Statement The mission of the Lake County Schools is to provide every student with individual opportunities to excel. Lake County Schools is committed to excellence in all curricular opportunities and instructional best practices. This focus area addresses closing the achievement gap, increased graduation rate, decreased dropout rate, increase in Level 3 and above scores on the FCAT, achieving an increase in the number of students enrolled in advanced placement and dual enrollment opportunities and implementing the best practices in instructional methodology. Summer Leadership Institute

21st Century Skills Tony Wagner, The Global Achievement Gap Critical Thinking and Problem Solving Collaboration and Leadership Agility and Adaptability Initiative and Entrepreneurialism Effective Oral and Written Communication Accessing and Analyzing Information Curiosity and Imagination Critical Thinking and Problem Solving: To compete in the new global economy, companies need their workers to think about how to continuously improve their products, processes, or services. “The challenge is this: How do you do things that haven't been done before, where you have to rethink or think anew? It's not incremental improvement any more. The markets are changing too fast.” Collaboration and Leadership: Teamwork is no longer just about working with others in your building. “Technology has allowed for virtual teams. We have teams working on major infrastructure projects that are all over the U.S. On other projects, you're working with people all around the world on solving a software problem. Every week they're on a variety of conference calls; they're doing Web casts; they're doing net meetings.” Agility and Adaptability: Ability to think, be flexible, change, and use a variety of tools to solve new problems. “We change what we do all the time. I can guarantee the job I hire someone to do will change or may not exist in the future, so this is why adaptability and learning skills are more important than technical skills.” Initiative and Entrepreneurialism: Taking chances and being a risk-taker. “I say to my employees, if you try five things and get all five of them right, you may be failing. If you try 10 things, and get eight of them right, you're a hero.” Effective Oral and Written Communication: The ability to be clear, concise, focused, energetic and passionate around the points they want to make. “We are routinely surprised at the difficulty some young people have in communicating: verbal skills, written skills, presentation skills. They have difficulty being clear and concise; it's hard for them to create focus, energy, and passion around the points they want to make. If you're talking to an exec, the first thing you'll get asked if you haven't made it perfectly clear in the first 60 seconds of your presentation is, ‘What do you want me to take away from this meeting?’ They don't know how to answer that question.” Accessing and Analyzing Information: The ability to know how to access and analyze large quantities of information. “There is so much information available that it is almost too much, and if people aren't prepared to process the information effectively it almost freezes them in their steps.” Curiosity and Imagination: The development of young people's capacities for imagination, creativity, and empathy will be increasingly important for maintaining the United States' competitive advantage in the future. “People who've learned to ask great questions and have learned to be inquisitive are the ones who move the fastest in our environment because they solve the biggest problems in ways that have the most impact on innovation.” Summer Leadership Institute

High Effect Size Indicators “The Department’s identified set of indicators on high effect size instructional and leadership strategies with a causal relationship to student learning growth constitute priority issues for deliberate practice and faculty development.” -Florida Department of Education, 2012 Student learning needs and faculty and leadership development needs will vary from school to school and from district to district. However, contemporary research reveals a core of instructional and leadership strategies that have a higher probability than most of positively impacting student learning in significant ways. The indicators below link formative feedback and evaluation to contemporary research on practices that have a positive impact on student learning growth. • Research on the cause and effect relationships between instructional and leadership strategies and student outcomes address the effect size of a strategy: What degree of impact does it have? • In the context of district instructional and leadership evaluation systems, effect size is a statistical estimation of the influence a strategy or practice has on student learning. Effect size calculations result from statistical analyses in research focused on student learning where the correct and appropriate use of a strategy yields better student learning growth than when the strategy is not used or is used incorrectly or inappropriately. • In research terms, those strategies often identified as “high effect size” are those with higher probabilities of improving student learning. Classroom teachers need a repertoire of strategies with a positive effect size so that what they are able to do instructionally, after adapting to classroom conditions, has a reasonable chance of getting positive results. As school leaders and mentor teachers begin to focus on feedback to colleagues to improve proficiency on practices that improve student learning growth, emphasis should be on those strategies that have a high effect size. Where every Florida classroom teacher and school leader has Summer Leadership Institute

Classroom Teacher High Effect Indicators School Leadership High Effect Indicators Learning Goal with Scales Tracking Student Progress Established Content Standards Multi-tiered System of Supports Clear Goals Text Complexity ESOL Students Feedback Practices Facilitating Professional Learning Clear Goals and Expectations Instructional Resources High Effect Size Strategies Instructional Initiatives Monitoring Text Complexity Interventions Instructional Adaptations ESOL Strategies Summer Leadership Institute

The Challenge: Florida State Statute 1008.22(8) (a) Measurement of the learning gains of students in all subjects and grade levels other than subjects and grade levels required for the state student achievement testing program is the responsibility of the school districts. (b) Beginning with the 2014-2015 school year, each school district shall administer for each course offered in the district a student assessment that measures mastery of the content, as described in the state-adopted course description, at the necessary level of rigor for the course. Such assessments may include: 1. Statewide assessments. 2. Other standardized assessments, including nationally recognized standardized assessments. 3. Industry certification examinations. 4. District-developed or district-selected end-of-course assessments.

Statewide Gap Analysis: Master Assessment Plan As of 2011, 2882 total courses offered K-12 statewide 636 with planned or existing statewide standardized assessments, FL Interim Assessments, District-developed Hard to Measure assessments, or other assessments (AP, IB, AICE, etc.) 2246 “gap” courses

District Gap Analysis Focus on 9-12 courses Lake County: 91 “gap” courses grades 9-12 with enrollments >50

Central Florida Assessment Collaborative “Gap” Courses Identified Physical Science (R&H) Anatomy & Physiology (R&H) Marine Science 1 & 2 (R&H) Integrated Science 1 & 2 Environmental Science Forensic Science Zoology Genetics Intensive Math* Pre-Calculus* Liberal Arts Math* Informal Geometry* Analysis of Functions* Adv Alg w/ Fin Applications* Analytic Geometry* Math for College Readiness Trigonometry* Calculus* Probability & Statistics* Anthropology* World Cultural Geography Psychology 1 & 2* Sociology* Law Studies American Economic Experience: Scarcity & Choice Civics (HS) Digital Design 1* Computers for College & Careers Creative Writing 1-5 Journalism 1-5* Chinese 1-4 Japanese 1-4 Russian 1-2 Arabic 1-4

CFAC Partner Districts Bay County Brevard County Charlotte County Citrus County Desoto County Duval County Glades County Hardee County Hendry County Hernando County Jackson County Lake County Leon County Levy County Manatee County Martin County Okeechobee County Orange County Osceola County Pasco County Pinellas County Polk County Sarasota County Seminole County Volusia County

Compensation for Participants CFAC Recommended compensation guidelines: $750/item specification developer $25/accepted item for item writers 4 items/hour reviewed by item reviewers, at LCEA agreed-upon hourly stipend

Item Specification Development: Step 1 – Identification of Assessable Benchmarks

Item Specification Development: Step 2 – Identification of Reporting Categories Example: Physical Science Physics Chemistry Human Impacts Nature of Science

Item Specification Development: Step 3 – Building out Item Specifications

Item Specification Development: Step 3 – Building out Item Specifications

Item Specification Development: Step 3 – Building out Item Specifications

Item Specification Development: Step 4: Designing the Assessment

Item Specification Development: Step 4: Designing the Assessment

Lake County Participants To Date: Gap Course Project: Hard to Measure Project: EHS ERHS LHS LMHS SLHS THS UHS MDHS 4 6 5 2 3 EHS ERHS LHS LMHS SLHS THS UHS MDHS 1 2 4

Future Opportunities for Involvement: “Gap” Science, Math, ELA, Social Studies, and CTE: Need additional item writers & reviewers Round 2 call for item writers & reviewers for Hard to Measure projects Possible Round 2 call for item writers & reviewers for Interim Assessment Item Bank

Participant Scale and Reflection (Please complete and turn in) 0-Not Using No understanding or implementation steps taken away 1-Beginning Little understanding and inconsistent implementation steps taken away 2-Developing Moderate understanding and implementation steps taken away 3-Applying Consistent understanding and implementation steps taken away along with monitoring componets for effective execution 4-Innovating In addition to criteria of Applying, enhanced understanding, implementation, monitoring, and execution take aways Summer Leadership Institute