Nebraska State Accountability (NeSA) Check for Learning (C4L) Item Writing Session October 2011

Get Connected  Choose wireless: neccguest  Login: necc  Password:  Go to:

Check 4 Learning A state generated item bank of Multiple choice items in reading, mathematics, and science that are Stored in the Computerized Assessment and Learning (CAL) System and Used to generate interim assessments to support the NeSA tests.

How Can C4Learning be used? Teachers, principals, or central office staff can pull items from the item bank to Generate mini-tests for use at point of instruction so that A check for learning can occur at Any time throughout the year.

We are here today to write test items for C4L! We will work together to assist and support one another. Our collaboration will ensure that all schools have their 10 items ready!

Get to know your teammates.  Share with your table mates:  Name  School  Position  Two things about yourself

Important Information... o Agenda/materials o Facilities o Internet Access o Cell phones o Questions o Special Introductions THANK YOU FOR ENGAGING IN THIS PROCESS

Attention and Refocus Signal  I will move to the front and center of the room.  Remaining time in the activity will be given.  Last 10 seconds will be counted down for wrap-up of your conversations.

A Quality Test... o Accurately and appropriately measures what is relevant. o Is consistent and precise. o Does not put any group at a disadvantage. Validity Fairness Reliability

Definition of Validity  Test validity refers to the degree to which the test actually measure what it claims to measure.

Definition of Reliability  Test reliability refers to the degree to which a test is consistent and stable in measuring what it is intended to measure.  Most simply put, a test is reliable if it is consistent within itself and across time.  How do we ensure reliability?  Follow item-writing guidelines  Provide several measurement opportunities

Fairness  Do the test scores mean the same thing for any test participant regardless of subgroup membership?  Gender  Ethnicity  Socioeconomic status  ELL

C4L... a statewide item bank!

 Is a scale of cognitive demand  Requires looking at the assessment item/standard – not the student  Context of the item/standard must be considered and the depth of thinking required, not just what verb was chosen Depth of Knowledge

Same Verb—Three Different DOK Levels DOK 1- Describe three characteristics of metamorphic rocks. (Simple recall) DOK 2- Describe the difference between metamorphic and igneous rocks. (Requires cognitive processing to determine the differences in the two rock types) DOK 3- Describe a model that you might use to represent the relationships that exist within the rock cycle. (Requires deep understanding of rock cycle and a determination of how best to represent it)

16 Table of Specifications

Item Difficulty Difficulty is a reference to how many students answer a question correctly. Easy: Most all students will get the right answer. Medium: About ½ the students will get the right answer. High: Few students will get the right answer.

Item Difficulty and Depth of Knowledge Important Item Information: o Key --- A o Indicator --- SC8.1.1.i o Difficulty Level --- Medium o Depth of Knowledge --- 2

Example: DOK, Item Difficulty In paragraph 8, which type of figurative language is the phrase “smooth as silk”? A. simile B. metaphor C. idiom D. Homophone DOK: Item Difficulty:

Example: DOK, Item Difficulty Which property of mathematics is shown in the equation? 2  (3  4) = (2  3)  4 A. distributive property B. multiplication property of 1 C. associative property of multiplication D. commutative property of multiplication DOK: Item Difficulty:

Example: DOK, Item Difficulty What could be another title for the passage? A. African American Architecture B. Father of California Architecture C. Architecture in the 20 th Century D. Architecture Found in California DOK: Item Difficulty:

Reading Passages... o Each participating district received two passages per grade level requested --- one narrative and one informational. o Highlight the standards on the TOS that apply only to informational text. o Write 5 items for narrative, 5 for informational text.

Numbers Related to Graphics... o Reading --- no items will have graphics. o Mathematics --- up to half of your items may include a graphic(s). (10 items/5 graphics) o Science --- use graphics sparingly --- up to 20 percent of your items may include a graphic(s). (10 items/2 graphics)

What is a Graphic? o In Check 4 Learning, a graphic is anything that requires more than a keystroke. = is NOT a graphic ⅜ IS a graphic 23 is NOT a graphic ÷ IS a graphic

The graphics library...

When Saving Items and Graphics... o Use the coding system explained on the next two slides. o Make sure there are no spaces in the code. o When using a graphic that is in the library, use the graphic name (shown below the thumbnail of the graphic) to identify the desired graphic in the template.

** Must be in.gif format

Item Writing Guidelines o Content o Style o Item Stems o Response Options

Item Writing Guidelines - CONTENT o Should be clearly aligned to the identified indicator o Should not reference material or information presented in another item o Should focus on one idea or concept o Should test meaningful content

Item Writing Guidelines - STYLE o Spell out acronyms o Minimize the use of “NOT” and “EXCEPT” o Avoid absolutes (e.g., always, never) and vague modifiers (e.g., best, worst) o Use appropriate vocabulary o Ensure correct grammar, punctuation, capitalization, spelling o Use active voice and present tense

Item Writing Guidelines – ITEM STEMS o State clearly and concisely o Use complete sentences with ending punctuation in context o Question appears at the end of the stem o Use which before a noun and what before a verb o Avoid negatives (capitalize if used) o Do NOT use “Which of the following... “

Item Writing Guidelines – ITEM RESPONSES o Homogeneous in terms of grammar, vocabulary, and text length o Logically consistent with the stem o Mutually exclusive (independent) o Phrase choices positively (avoid NOT) o Do NOT use “All (or none) of the above” o Avoid cues to correct choice o Ordered purposefully (by length, alphabetically, etc.) o Are all plausible

Example The author stated the dragonfly had “lace-like wings” which means what? A. The wings allow them to hover B. The wings enable them to fly many miles C. The wings have delicate patterns D. The wings have rainbow colors o DOK: Item Difficulty:

Example Which is the main function of skin? A. Protects from infection B. Breaks down food C. Enables movement D. Moves blood DOK: Item Difficulty:

Example What detail supports the heading Lilac Varieties? A. Two presidents wrote about lilacs B. Lilacs grow in parks C. Lilacs have purple, pink, or white D. Lilacs bloom in time for May Day DOK: Item Difficulty:

Write Items  Independently write 3-5 items to varying indicators.  Work with a partner  Read each other’s items  Offer suggestions to enhance independently written items  High quality items  Align to Indicator  DOK Level  Item Difficulty  Item Guidelines

THANK YOU!