1 Bureau of K-12 Student Assessment District Assessment and Accountability Coordinators Annual Meeting

2 Introductions Bureau of K-12 Student Assessment Administration Susie Lee, Director Catherine Altmaier Jenny Black Kira Bland Lisa Nash Becky Stephens 2

3 Introductions Bureau of K-12 Student Assessment Scoring and Reporting Dr. Qian Liu, Director Gabriel Colorado Marilyn Dwyer 3

4 Introductions American Institutes for Research (AIR) Kevin Murphy, Director, State Programs Jeff Burger, Technical Director Ghi Crisafulli, Assessment Communications Manager Danni Greenberg, Program Manager Eugenia Kim, Senior Score Reporting Project Manager Margaret Won, Scoring & Reporting Specialist Data Recognition Corporation (DRC) Pete Tressel, Director, State Programs Angie Karn, Senior Project Manager Mike Kulzer, Senior Project Manager 4

5 Introductions Pearson Andy Messenger, Vice President, State Services Chad Curtis, Senior Program Manager, Scoring & Reporting Leann Spaight, Program Manager, Computer-Based Testing Amanda Pepmiller, Program Manager, Test Development Paul Arens, Director, Online Testing/Core Engineering 5

6 Question Form If you have questions during the meeting today that are not addressed, you may submit them in a Google form. cJKqkJvztI54LIkkeHTeqxD3FPW6NpU/viewform cJKqkJvztI54LIkkeHTeqxD3FPW6NpU/viewform 6

7 Communication Monthly Summary—A Success! Provides a list of all communications sent from our office in the previous month. Provides a PDF file of all communications. Provides important dates for the next month. Provides contact information for the K-12 Student Assessment team members. 7

Standard Setting Updates 2014 Civics End-of-Course Assessment Standard Setting Update

9 Standard Setting: A Multi-Stage Process

10 Important Dates Achievement Level Description (ALD) Panel: April 28–May 1 Hotel Duval, Tallahassee, Florida Educator Panel: August 31–September 4, 2015 Grand Cypress, Orlando, Florida Reactor Panel: September 10–11, 2015 Grand Cypress, Orlando, Florida

11 Achievement Level Description Panel April 28–May 1, 2015 Four-day workshop Forty-two panelists Described achievement levels for content standards by grade and subject

12 Standard Setting Educator Panel August 31–September 4, 2015 Five-day standard-setting workshop Four rounds of standard setting About 300 panelists Seventeen rooms setting standards concurrently Recommending cut scores based primarily on content, though impact and benchmark data is reviewed after the first two rounds of judgments

13 Standard Setting Reactor Panel September 10–11, 2015 Two-day meeting composed of community/education organization leaders state university leaders business leaders school board members superintendents Review educator panel recommended cut scores and impact data Two rounds of judgment About 20 panelists

14 Standard setting becomes necessary whenever any of the following occur: New test Curriculum updates Blueprint changes ALDs change FSA: new assessments for adopted content standards When Is Standard Setting Necessary?

15 A process of deriving levels of performance on educational or professional assessments, by which decisions or classifications of persons will be made (Cizek, 2006) Test scores can be used to group students into meaningful achievement levels Standard setting is the process whereby we draw the lines that separate the test scores into various achievement levels What Is Standard Setting?

16 Standard-Setting Method Bookmark Method—One of a number of approaches available for standard setting One of the most widely used and most defensible methods Research-based procedure Used in many state assessment programs Proven to be technically sound in litigation Mitzel, H. C., Lewis, D. M., Patz, R. J. & Green, D. R. (2001). The Bookmark procedure: Psychological perspectives. In: G. Cizek (Ed.), Setting performance standards: Concepts, methods and perspectives. Mahwah, NJ: Erlbaum.

17 We’ve Done This Before… 1998: Reading and Mathematics Achievement Levels approved for grades 4, 5, 8, and : Reading and Mathematics Achievement Levels approved for grades 3–10 Grade 10 FCAT Reading and Mathematics passing scores established 2011: FCAT 2.0 Reading (grades 3–10) and Mathematics (3–8) Achievement Levels approved Algebra 1 EOC Assessment Achievement Levels approved Grade 10 FCAT 2.0 Reading and EOC assessment passing scores established in rule as the minimum score in Achievement Level : FCAT 2.0 Science (grades 5 and 8) Achievement Levels approved Biology 1 and Geometry EOC Assessment Achievement Levels approved 2013: U.S. History EOC Assessment Achievement Levels approved 2014: Civics EOC Assessment Achievement Levels approved

18 Standard Setting – Next Steps Rule Development Workshops Broward County – September 15 Orange County – September 16 Leon County (Webcast) – September 17 Rule workshop information and public feedback form will be available on the FDOE website at student-assessment/stard-setting.stml student-assessment/stard-setting.stml

THANK YOU!