2. An Overview of Virginia Standards of Learning Item and Test Development

3. Purpose of this Presentation  Describe the Item and Test Review Committees  Provide an overview of Standards of Learning (SOL) Item and Test Development

4. Item and Test Review Committees   On an annual basis, Virginia educators from across the state participate in the development of the Virginia Standards of Learning (SOL) assessments.   Every summer, Item and Test Review Committees convene in the Richmond area to review test items and test forms for the SOL assessment program.

5. Item and Test Review Committees   Content committees are composed primarily of educators teaching the subject of the test, as well as instructional specialists or other curriculum staff.   Committee members represent a geographical cross-section of Virginia.

6. Item and Test Review Committees   New members are added each year in order to provide for a balance of experienced educators and new members, and to bring new perspectives into committee meetings.

7. Item and Test Review Committees Committee members are selected based on the following criteria:   grade-level experience and content area expertise;   in-depth knowledge of the SOL;   instructional/supervisory experience with students of varied learning styles, abilities, and aptitudes, including students with disabilities and students with limited English proficiency; and   balanced regional representation.

8. Item and Test Review Committees The Item and Test Review Committees play an important and essential role at three major stages of the Test Development Process:   review of new field-test items;   data review of field-tested items; and   review of test forms. An overview of the Item and Test Development Process follows.

9. Overview of Steps in Test Development  Item is Written: The test item is developed by Virginia’s testing contractor.  Item Review: The new test item is reviewed by an Item and Test Review Committee, comprised of Virginia educators. If approved, the new item can be field-tested on a Standards of Learning (SOL) test.  Field-Testing: The new item is placed on an SOL test and tried out with Virginia students.  Data Review: The item with its field-test data is reviewed again by an Item and Test Review Committee, comprised of Virginia educators. If approved, the item can be used on a test as an item that counts towards a student’s score.  Test Forms Review: Newly constructed tests are reviewed by the educators on the Item and Test Review Committee.

10. Item Development Overview Item Writing Phase The testing contractor develops items:  Item writers are trained specifically to interpret the SOL while using the Curriculum Framework.

11. Item Development Overview Item Writing Phase (continued) Items are reviewed by the testing contractor for:   content accuracy and correct answer;   match to the SOL; and   bias and sensitivity (fairness for all groups): an item is free from prejudice, favoritism or stereotyping regarding gender, ethnic, religious, political, age, or socio-economic group; there is no “advantage” or “disadvantage” for any one group over other groups of students.

12. Item Review Phase Educators on the Item and Test Review Committee examine and judge each item for:  content accuracy and correct answer;  its match to the SOL;  appropriateness; and  fairness to all groups. Item Development Overview

13.  Items accepted by the committee are eligible for field-testing.  Newly approved items are tried out by Virginia students.  Field-test items are included in the SOL tests but are not counted in student scores. Field-Testing Phase Item Development Overview

14. Data Review Phase  Student performance on items is described by field-test statistical results.  The Virginia Item and Test Review Committee examines field-tested items for a second time and reviews field-test data.  The committee makes judgments about items based on:  match to the SOL;  appropriateness;  fairness to all groups; and  field-test data.

15. Test Development Overview Test Form Construction Phase  The testing contractor uses items approved by the Item and Test Review Committee to construct new tests.  Newly constructed tests must match the test blueprint and meet statistical criteria.  The Item and Test Review Committee examines new tests before they are administered.

16. Test Development Overview  Tests are administered to Virginia students and scored using only the approved operational (non-field-test) items.  All tests for a given SOL test are statistically equated and score results are produced. This is a very important step to ensure that students are required to demonstrate the same level of proficiency regardless of which test form they take. That is, scores carry the same meaning regardless of which test form a student takes. Only then are test results produced and sent to the divisions throughout the state.

17. For further questions, contact student_assessment@doe.virginia.gov student_assessment@doe.virginia.gov