End Of Course Assessments Field Test Spring 2008 By The Missouri Department of Elementary and Secondary Education Assessment Section April 2008

PowerPoint Overview This presentation is geared toward the Spring 2008 Field Test and contains the following information: A brief history of End-Of-Course (EOC)A brief history of End-Of-Course (EOC) EOC guidelines and calibrations with No Child Left Behind (NCLB)EOC guidelines and calibrations with No Child Left Behind (NCLB) An overview of the test development processAn overview of the test development process The design of the EOC Field TestThe design of the EOC Field Test The EOC BlueprintsThe EOC Blueprints Information about ancillaries and reference sheetsInformation about ancillaries and reference sheets

Why End of Course? DESE responded to feedback from Missouri districts regarding large scale assessments for secondary school. DESE responded to feedback from Missouri districts regarding large scale assessments for secondary school. EOC Assessments were created to adapt testing to the needs of Missouri districts, schools, teachers and students while meeting state and federal requirements. EOC Assessments were created to adapt testing to the needs of Missouri districts, schools, teachers and students while meeting state and federal requirements.

EOC Assessments were created in conjunction with national standards of excellence and Show- Me Standards. Both Course Level Expectations (CLE’s) and the EOC Assessments are a continuous collaborative effort with: Missouri teachers Administrators Regional Instructional Facilitators The Missouri Department of Elementary and Secondary Education (DESE) Assessment Resource Center (ARC) The Riverside Publishing Company

Missouri educators are playing a key role in the development of EOC assessments: Authoring itemsAuthoring items Reviewing and revising items for content, sensitivity, and biasReviewing and revising items for content, sensitivity, and bias Revising items, when neededRevising items, when needed Approving items to be on the final operational test forms.Approving items to be on the final operational test forms.

Specific Guidelines In order to ensure the best assessments possible, the advisory committee and DESE have set specific guidelines for the tests. The items must: 1.Be in full compliance with the federal legislation, No Child Left Behind (NCLB) 1.Be aligned to the Missouri Course Level Expectations (CLE’s) 2.Be aligned to Norman Webb’s Depth of Knowledge (DOK) cognitive levels 3.Include multiple choice and performance events. (Algebra I, Biology, and English II are the only tests that will have performance events). 4.Have both stand-alone and passage-based items 5.Be criterion-referenced tests

Missouri State Board of Education The Missouri State Board of Education identified the following purposes for EOC: Measuring and reflecting student mastery toward post-secondary readiness Measuring and reflecting student mastery toward post-secondary readiness Identifying students’ strengths and weaknesses Identifying students’ strengths and weaknesses Communicating expectations for all students Communicating expectations for all students Serving as the basis of state and national accountability plans Serving as the basis of state and national accountability plans Evaluating programs Evaluating programs

CLE’s instead of GLE’s Course Level Expectations are goals for a particular course rather than a grade level. Course Level Expectations are goals for a particular course rather than a grade level. A course such as Algebra I could be delivered in 8 th grade or any grade level in secondary school. A course such as Algebra I could be delivered in 8 th grade or any grade level in secondary school. Districts can offer courses with different titles which cover the same CLE’s. Districts can offer courses with different titles which cover the same CLE’s.

The right test at the right time The responsibility for testing students belongs to the school district. The responsibility for testing students belongs to the school district. The district has the responsibility to match its course content with the CLE’s that are being tested in the EOC Assessment. The district has the responsibility to match its course content with the CLE’s that are being tested in the EOC Assessment. When the content of the assessed information is covered, the test can be administered regardless of student grade level. When the content of the assessed information is covered, the test can be administered regardless of student grade level. Examiners will deliver the EOC assessment at the appropriate time for their students. Examiners will deliver the EOC assessment at the appropriate time for their students.

1 st Field Test Spring 2008 Algebra I Biology English II

A full census field test will be administered in three content areas in approximately 2000 Missouri schools in the Spring of The first operational administration of EOC will follow during the school year.

Online Version A small group of students will trial the online version of the Spring 2008 EOC Field Test. A small group of students will trial the online version of the Spring 2008 EOC Field Test. An online version of each test will also be available in the operational test to school districts wishing to use this format. An online version of each test will also be available in the operational test to school districts wishing to use this format.

No reports will be produced for the districts from this Spring 2008 Field Test. No reports will be produced for the districts from this Spring 2008 Field Test. The EOC Assessment Field Test will be used to provide future operational tests. The EOC Assessment Field Test will be used to provide future operational tests. District Reports

Braille/Large Print For the field test, the EOC will not have a Braille/Large Print edition. Therefore, students needing Braille/Large Print editions may have all content areas, including the English II assessment, read to them without invalidating the test. For the field test, the EOC will not have a Braille/Large Print edition. Therefore, students needing Braille/Large Print editions may have all content areas, including the English II assessment, read to them without invalidating the test. The Examiner must mark the new bubble on the SIS for the above accommodation. The Examiner must mark the new bubble on the SIS for the above accommodation.

Forms EOC Field Test consists of ten forms for both Sessions I and II of testing. EOC Field Test consists of ten forms for both Sessions I and II of testing. The forms are spiraled (distributed in roughly equal numbers) within each content area and should not be separated. The forms are spiraled (distributed in roughly equal numbers) within each content area and should not be separated.

Two Sessions The EOC Field Tests are not strictly timed and are designed to fit into a normal class period. EOC Field Test consists of two sessions for each content area Session I = 55 minutes Session I = 55 minutes Session II = 55 minutes Session II = 55 minutes 55-minute class period (times are approximate) 10 minutes for test distribution, directions, collection, etc. and10 minutes for test distribution, directions, collection, etc. and 45 minutes for actual test taking45 minutes for actual test taking

Session I Selected-response items (also known as multiple choice questions) Selected-response items (also known as multiple choice questions) Each item (question) has four response options – one correct answer and three plausible distracters Each item (question) has four response options – one correct answer and three plausible distracters

Session II Performance Event Performance Event Requires students to work through more complicated itemsRequires students to work through more complicated items Makes allowances for more than one correct answerMakes allowances for more than one correct answer Allows for insight to be garnered about the student’s ability to apply their knowledge and understanding to real- life situationsAllows for insight to be garnered about the student’s ability to apply their knowledge and understanding to real- life situations

Writing Prompt This is a special type of performance event that appears in the English II assessment This is a special type of performance event that appears in the English II assessment It is open-ended It is open-ended It is scored holistically It is scored holistically

Field Test Specifications Essential content and statistical considerations are aligned to: Depth of Knowledge cognitive levelsDepth of Knowledge cognitive levels Content Strand/Course Level ExpectationsContent Strand/Course Level Expectations

Algebra I Field Test Specifications 20 Forms 20 Forms 10 multiple-choice forms each with 18 items10 multiple-choice forms each with 18 items 10 performance-event forms each with one performance event10 performance-event forms each with one performance event Each student being field tested will take:Each student being field tested will take: 1 multiple-choice form (Session I) and 1 multiple-choice form (Session I) and 1 performance event form (Session II) 1 performance event form (Session II)

Field to Operational Field Tested Items will be reviewed and selected for the Operational Test based on: Content appropriatenessContent appropriateness Statistical performance (Appendix A)Statistical performance (Appendix A) Depth of Knowledge alignmentDepth of Knowledge alignment

Operational Test The field test items will be processed into 4 operation forms for the Operational EOC Assessment. Each operational form will consist of 2 sessions: Session I = 35 multiple-choice itemsSession I = 35 multiple-choice items Session II = 1 performance eventSession II = 1 performance event

Operational Algebra I Blueprint Content Strand Target # Points 10% Toleranc e Point Range Target % Total Points Min Emphasis Max Emphasis Number & Operation %19%23% Algebraic Relationships 23 INCLUDES A 4 POINT PERFORM- ANCE EVENT %53%63% Geometric & Spatial Relationships 0000%0%0% Measurement0000%0%0% Data & Probability %19%23% TOTALS39100%

Guiding Principles for Test Construction Each performance event will be aligned to one CLE from the Algebraic Relationships strand. Each performance event will be aligned to one CLE from the Algebraic Relationships strand. On the operational tests, a balance of items written for particular CLEs will be sought. On the operational tests, a balance of items written for particular CLEs will be sought. Particular CLEs may be only assessed once on a form while other forms may contain several items for that CLE. Particular CLEs may be only assessed once on a form while other forms may contain several items for that CLE. Some CLEs may be assessed more frequently because the skills are integral to the course. Some CLEs may be assessed more frequently because the skills are integral to the course.

Operational English II Blueprint Content Strand Target # Points Point Range Target % Total Points Min Emphasis Max Emphasis Develop and apply skills and strategies to the reading process %26%36% Develop and apply skills and strategies to comprehend, analyze and evaluate fictions, poetry and drama %23%28% Develop & apply skills & strategies to comprehend, analyze & evaluate nonfiction %23%28% Writing: Apply a writing process in composing text Compose well- developed text Write effectively in various forms & types of writing 9* *This strand includes the 4-point writing prompt %23%23% TOTALS39100%

Guiding Principles for Test Construction Each form will contain 5 passages: Either 2 fiction passages and 3 non-fiction passages or vice versa. Each form will contain 5 passages: Either 2 fiction passages and 3 non-fiction passages or vice versa. The writing form/type will vary depending on the writing prompt selected. Writing prompts will be aligned to a primary CLE. Multiple writing CLES may be assessed to reflect the holistic rubric. The writing form/type will vary depending on the writing prompt selected. Writing prompts will be aligned to a primary CLE. Multiple writing CLES may be assessed to reflect the holistic rubric. Some CLEs can be used for both fiction and nonfiction passages. Some CLEs can be used for both fiction and nonfiction passages. Particular CLEs may be only assessed once on a form while other forms may contain several items for that CLE. Particular CLEs may be only assessed once on a form while other forms may contain several items for that CLE. The goal will be to balance the CLEs across the forms. Since English II is passage-based, including every assessable CLE in this round of operational forms may not be possible. The goal will be to balance the CLEs across the forms. Since English II is passage-based, including every assessable CLE in this round of operational forms may not be possible.

Operational Biology Blueprint Content Strand Target # Points 10%Tolerance Point Range Target % Total Points Min Emphasis Max Emphasis 1.Properties and Principle of Matter and Energy. 0000%0%0% 2.Properties and Principles of Force and Motion. 0000%0%0% 3.Characteristic and Interactions of Living Organisms %36%44% 4.Changes in Ecosystems and Interactions of Organisms with their Environments %22%27% 5.Processes and Interactions of the Earth’s systems (Geosphere, Atmosphere, & Hydrosphere) 0000%0%0% Continued on next slide

Operational Biology Blueprint Continued from previous slide Content Strand Target # Points 10% Tolerance Point Range Target % Total Points Min Emphasis Max Emphasis 6.Composition and Structure of the Universe and the Motion of the Objects within it. 0000%0%0% 7. Scientific Inquiry 20* This strand contains the performance event (2.0)2036%36%36% 8. Impact of Science, Technology and Human Activity 0.000%0%0% TOTALS55100%

Breakdown by DOK for all content areas DOK levels for each CLE were determined by groups of Missouri educators during an alignment review, facilitated by Norman Webb. DOK levels for each CLE were determined by groups of Missouri educators during an alignment review, facilitated by Norman Webb. At least 80% or more of the items on the operational form should be at the designated DOK level. At least 80% or more of the items on the operational form should be at the designated DOK level. The other 20% may fall below the designated DOK level. The other 20% may fall below the designated DOK level.

Non-secure materials Non-secure materials will be delivered to the districts the week of April 14 th Non-secure materials will be delivered to the districts the week of April 14 th Non-secure materials include: Test Coordinator Manuals, Examiners Manuals, barcode labels, return kits, and answer documents for Session I Assessments Non-secure materials include: Test Coordinator Manuals, Examiners Manuals, barcode labels, return kits, and answer documents for Session I Assessments

Manipulatives for the field test Algebra I: Algebra I: Students can use a standard four-function calculator during both Session I and Session II.Students can use a standard four-function calculator during both Session I and Session II. The testing materials include an Algebra I reference sheet for use during both Session I and Session II.The testing materials include an Algebra I reference sheet for use during both Session I and Session II. Biology: Biology: Students are not provided nor do they need a reference sheet.Students are not provided nor do they need a reference sheet. Students can NOT use a calculator.Students can NOT use a calculator. English II: English II: Students can use a standard dictionary, thesaurus, orStudents can use a standard dictionary, thesaurus, or grammar handbook only for Session II. NOTE: For students taking either the Algebra I or Biology test, their writing, spelling and grammar will not be evaluated.

Labels Each content area will have 2 labels, one for Session I and the second for Session II Each content area will have 2 labels, one for Session I and the second for Session II The labels will be pre-coded based on the MOSIS core data The labels will be pre-coded based on the MOSIS core data The labels are sorted by District, School, Grade, Last Name, First Name The labels are sorted by District, School, Grade, Last Name, First Name

Secure Materials Session I and Session II test books will arrive in the Districts the week of April 21st Session I and Session II test books will arrive in the Districts the week of April 21st

Rules The rules for EOC testing are basically the same as MAP testing regarding calculators, highlighters, erasures, all testing environments, etc. The rules for EOC testing are basically the same as MAP testing regarding calculators, highlighters, erasures, all testing environments, etc. The same rules apply for test security and test administration in the EOC as in MAP. The same rules apply for test security and test administration in the EOC as in MAP.

Costs No cost for the EOC Field Test No cost for the EOC Field Test The Operational EOC Assessment is: The Operational EOC Assessment is: $1.80 per content Algebra I, Biology, and English II.$1.80 per content Algebra I, Biology, and English II. Other EOC Assessments when offered will be free of charge (at this time).Other EOC Assessments when offered will be free of charge (at this time).

Glossary Open-ended question - the student’s answer can vary depending on the student’s thought process. An example of an open-ended item might be to write a friendly letter to a friend telling them about the sports event you just watched Open-ended question - the student’s answer can vary depending on the student’s thought process. An example of an open-ended item might be to write a friendly letter to a friend telling them about the sports event you just watched Holistically scored – Scored on the overall impression of how well a response reflects the scoring criteria. Holistically scored – Scored on the overall impression of how well a response reflects the scoring criteria.