1. Summer Leadership Institute
New Test Item Specifications for High School Courses: Implications for Upcoming Common Assessments and Opportunities for Involvement
Heather P. Wright
August 9-10, 2012

2. Common Board Configuration
Date: August 9 & 10, 2012
Vocabulary: Item Specifications
Bell Ringer: How do your teachers use item specifications?
Agenda: Gradual Release
Learning Goal:Understanding the multiple implications of new test item specifications for previously untested courses
Benchmark: N/A
Objective: Participants will understand the creation process for the new test item specification for high school “gap” courses, will understand the instructional implications of these item specifications, and will learn about upcoming opportunities for involvement in the “Gap” course assessment project.
Summarizing Activity: Consider how you will use test item specifications in your school to improve instruction.
Essential Question: What is the creation process for the test item specifications for high school “gap” courses? How do these test item specifications affect upcoming common assessments? What professional development opportunities and opportunities for involvement are available for Lake County teachers?
Homework: Encourage your teachers to participate as item writers and reviewers.

3. Lake County Schools Vision Statement
A dynamic, progressive and collaborative learning community embracing change and diversity where every student will graduate with the skills needed to succeed in postsecondary education and the workplace.
Mission Statement
The mission of the Lake County Schools is to provide every student with individual opportunities to excel.
Lake County Schools is committed to excellence in all curricular opportunities and instructional best practices. This focus area addresses closing the achievement gap, increased graduation rate, decreased dropout rate, increase in Level 3 and above scores on the FCAT, achieving an increase in the number of students enrolled in advanced placement and dual enrollment opportunities and implementing the best practices in instructional methodology.
Summer Leadership Institute

4. 21st Century Skills Tony Wagner, The Global Achievement Gap
Critical Thinking and Problem Solving
Collaboration and Leadership
Agility and Adaptability
Initiative and Entrepreneurialism
Effective Oral and Written Communication
Accessing and Analyzing Information
Curiosity and Imagination
Critical Thinking and Problem Solving: To compete in the new global economy, companies need their workers to think about how to continuously improve their products, processes, or services. “The challenge is this: How do you do things that haven't been done before, where you have to rethink or think anew? It's not incremental improvement any more. The markets are changing too fast.”
Collaboration and Leadership: Teamwork is no longer just about working with others in your building. “Technology has allowed for virtual teams. We have teams working on major infrastructure projects that are all over the U.S. On other projects, you're working with people all around the world on solving a software problem. Every week they're on a variety of conference calls; they're doing Web casts; they're doing net meetings.”
Agility and Adaptability: Ability to think, be flexible, change, and use a variety of tools to solve new problems. “We change what we do all the time. I can guarantee the job I hire someone to do will change or may not exist in the future, so this is why adaptability and learning skills are more important than technical skills.”
Initiative and Entrepreneurialism: Taking chances and being a risk-taker. “I say to my employees, if you try five things and get all five of them right, you may be failing. If you try 10 things, and get eight of them right, you're a hero.”
Effective Oral and Written Communication: The ability to be clear, concise, focused, energetic and passionate around the points they want to make. “We are routinely surprised at the difficulty some young people have in communicating: verbal skills, written skills, presentation skills. They have difficulty being clear and concise; it's hard for them to create focus, energy, and passion around the points they want to make. If you're talking to an exec, the first thing you'll get asked if you haven't made it perfectly clear in the first 60 seconds of your presentation is, ‘What do you want me to take away from this meeting?’ They don't know how to answer that question.”
Accessing and Analyzing Information: The ability to know how to access and analyze large quantities of information. “There is so much information available that it is almost too much, and if people aren't prepared to process the information effectively it almost freezes them in their steps.”
Curiosity and Imagination: The development of young people's capacities for imagination, creativity, and empathy will be increasingly important for maintaining the United States' competitive advantage in the future. “People who've learned to ask great questions and have learned to be inquisitive are the ones who move the fastest in our environment because they solve the biggest problems in ways that have the most impact on innovation.”
Summer Leadership Institute

5. High Effect Size Indicators
“The Department’s identified set of indicators on high effect size instructional and leadership strategies with a causal relationship to student learning growth constitute priority issues for deliberate practice and faculty development.”
-Florida Department of Education, 2012
Student learning needs and faculty and leadership development needs will vary from school to school and from district to district. However, contemporary research reveals a core of instructional and leadership strategies that have a higher probability than most of positively impacting student learning in significant ways. The indicators below link formative feedback and evaluation to contemporary research on practices that have a positive impact on student learning growth.
• Research on the cause and effect relationships between instructional and leadership strategies and student outcomes address the effect size of a strategy: What degree of impact does it have?
• In the context of district instructional and leadership evaluation systems, effect size is a statistical estimation of the influence a strategy or practice has on student learning. Effect size calculations result from statistical analyses in research focused on student learning where the correct and appropriate use of a strategy yields better student learning growth than when the strategy is not used or is used incorrectly or inappropriately.
• In research terms, those strategies often identified as “high effect size” are those with higher probabilities of improving student learning.
Classroom teachers need a repertoire of strategies with a positive effect size so that what they are able to do instructionally, after adapting to classroom conditions, has a reasonable chance of getting positive results. As school leaders and mentor teachers begin to focus on feedback to colleagues to improve proficiency on practices that improve student learning growth, emphasis should be on those strategies that have a high effect size. Where every Florida classroom teacher and school leader has
Summer Leadership Institute

6. Classroom Teacher High Effect Indicators
School Leadership High Effect Indicators
Learning Goal with Scales
Tracking Student Progress
Established Content Standards
Multi-tiered System of Supports
Clear Goals
Text Complexity
ESOL Students
Feedback Practices
Facilitating Professional Learning
Clear Goals and Expectations
Instructional Resources
High Effect Size Strategies
Instructional Initiatives
Monitoring Text Complexity
Interventions
Instructional Adaptations
ESOL Strategies
Summer Leadership Institute

7. The Challenge: Florida State Statute 1008.22(8)
(a) Measurement of the learning gains of students in all subjects and grade levels other than subjects and grade levels required for the state student achievement testing program is the responsibility of the school districts. (b) Beginning with the school year, each school district shall administer for each course offered in the district a student assessment that measures mastery of the content, as described in the state-adopted course description, at the necessary level of rigor for the course. Such assessments may include: 1. Statewide assessments. 2. Other standardized assessments, including nationally recognized standardized assessments. 3. Industry certification examinations. 4. District-developed or district-selected end-of-course assessments.

8. Statewide Gap Analysis: Master Assessment Plan
As of 2011, 2882 total courses offered K-12 statewide
636 with planned or existing statewide standardized assessments, FL Interim Assessments, District-developed Hard to Measure assessments, or other assessments (AP, IB, AICE, etc.)
2246 “gap” courses

9. District Gap Analysis Focus on 9-12 courses
Lake County: 91 “gap” courses grades 9-12 with enrollments >50

10. Central Florida Assessment Collaborative “Gap” Courses Identified
Physical Science (R&H)
Anatomy & Physiology (R&H)
Marine Science 1 & 2 (R&H)
Integrated Science 1 & 2
Environmental Science
Forensic Science
Zoology
Genetics
Intensive Math*
Pre-Calculus*
Liberal Arts Math*
Informal Geometry*
Analysis of Functions*
Adv Alg w/ Fin Applications*
Analytic Geometry*
Math for College Readiness
Trigonometry*
Calculus*
Probability & Statistics*
Anthropology*
World Cultural Geography
Psychology 1 & 2*
Sociology*
Law Studies
American Economic Experience: Scarcity & Choice
Civics (HS)
Digital Design 1*
Computers for College & Careers
Creative Writing 1-5
Journalism 1-5*
Chinese 1-4
Japanese 1-4
Russian 1-2
Arabic 1-4

11. CFAC Partner Districts
Bay County
Brevard County
Charlotte County
Citrus County
Desoto County
Duval County
Glades County
Hardee County
Hendry County
Hernando County
Jackson County
Lake County
Leon County
Levy County
Manatee County
Martin County
Okeechobee County
Orange County
Osceola County
Pasco County
Pinellas County
Polk County
Sarasota County
Seminole County
Volusia County

12. Compensation for Participants
CFAC Recommended compensation guidelines:
$750/item specification developer
$25/accepted item for item writers
4 items/hour reviewed by item reviewers, at LCEA agreed-upon hourly stipend

13. Item Specification Development: Step 1 – Identification of Assessable Benchmarks

14. Item Specification Development: Step 2 – Identification of Reporting Categories
Example: Physical Science
Physics
Chemistry
Human Impacts
Nature of Science

15. Item Specification Development: Step 3 – Building out Item Specifications

16. Item Specification Development: Step 3 – Building out Item Specifications

17. Item Specification Development: Step 3 – Building out Item Specifications

18. Item Specification Development: Step 4: Designing the Assessment

19. Item Specification Development: Step 4: Designing the Assessment

20. Lake County Participants To Date:
Gap Course Project:
Hard to Measure Project:
EHS
ERHS
LHS
LMHS
SLHS
THS
UHS
MDHS 4 6 5 2 3
EHS
ERHS
LHS
LMHS
SLHS
THS
UHS
MDHS 1 2 4

21. Future Opportunities for Involvement:
“Gap” Science, Math, ELA, Social Studies, and CTE: Need additional item writers & reviewers
Round 2 call for item writers & reviewers for Hard to Measure projects
Possible Round 2 call for item writers & reviewers for Interim Assessment Item Bank

22. Participant Scale and Reflection (Please complete and turn in)
0-Not Using
No understanding or implementation steps taken away
1-Beginning
Little understanding and inconsistent implementation steps taken away
2-Developing
Moderate understanding and implementation steps taken away
3-Applying
Consistent understanding and implementation steps taken away along with monitoring componets for effective execution
4-Innovating
In addition to criteria of Applying, enhanced understanding, implementation, monitoring, and execution take aways
Summer Leadership Institute