GUIDELINES for the Writing Assessments How to Write Test Items Professor Norland

Criterion-Referenced tests Teacher-made tests that have the objectives, or standards as the goals, can be called criterion-referenced tests. In other words, the test items are written to assess if the students have met the ‘criteria’ for the unit or lesson. Identify the test objectives and clearly stated these objectives or criterion for what you want your students to learn.

Writing test questions in this assessment practice would be considered a formal SUMMATIVE as if the students just completed a unit, and you are testing them on what they learned. “Summary” of students’ knowledge of the objectives (your criterion) according to how successful they are on your test items.

Make sure your test-taking directions are clear Make sure your test-taking directions are clear. Do not assume it is obvious what to do.

Choose to use simple sentences instead of complicated sentences. Don’t write test items that have unintended clues that help students guess the correct answer. The bird in the story was an ___? (eagle, hawk, robin) Choose to use simple sentences instead of complicated sentences. Make sure the vocabulary you use in the test is suitable to your students. Pay attention to the words you start questions with (what, who, when, where),which assess Knowledge, or (how, why), which require higher-levels of thinking.

SELECTED-RESPONSE ITEMS Selected-response requires that the students choose one correct answer per item. Students do not have to come up with their own answers; they instead are given options to select. •True/False, Multiple choice, and Matching items are selected-response items.

These may also be called BINARY-CHOICE because there are (2) options: TRUE-FALSE YES-NO RIGHT-WRONG FACT-OPINION

(Selected Response) TRUE-FALSE Phrase items thoughtfully, so that none are obviously true or false. Avoid using negative words in true-false items. King Tut was NOT a pharaoh of Egypt. Alligators are NEVER found in salt-water. Avoid using qualifiers such as “always”, “usually”, “often”, “seldom”, “sometimes”, “every”, “never”. Alligators OFTEN live in fresh-water. Boys USUALLY excel in math and engineering.

TRUE-FALSE Focus on only one idea in each statement so that students do not have to deal with the possible truth or falsity of two parts at once . The highest level of Bloom’s is evaluation and the lowest level is comprehension. Have a fairly equal amount of true-false answers in all. (OR use more false than true statements because students tend to mark true more often than they mark false when guessing.) Randomize the sequence of true and false statements.

TRUE-FALSE If a statement expresses a relationship, such as cause and effect, present the correct part of the statement first and vary the truth or falsity of the second part. Mixing bleach and vinegar will produce a toxic gas. (T) (1ST PART CORRECT ………..2ND PART FALSE) Lincoln’s election was the key to the North seceding. (F)

TRUE-FALSE Make true and false statements of approximately equal and average length throughout the test. (Longer statements are more likely to be true). •Write items with the important test objectives in mind; not simply a focus on memorization: recalling of specific details, numbers, dates, etc.

(Selected Response) MULTIPLE-CHOICE ITEMS Focuses on important concepts. (Consider test objectives!) Start out with a longer statement or question, which is called the “STEM.”

STEM: The first part of the item preceding the response options/choices or “stimulus material.”

MULTIPLE-CHOICE The STEM should have a self-contained question or problem, or statement: A main cause leading up to the Civil War was this: Which of the following (of the diagrams shown) is an isosceles triangle?

MULTIPLE-CHOICE Avoid negative words ( “not”, “never”) and “always” in the stem. It IS common and appropriate for Multiple Choice stems to use ‘qualifiers’ such as “probably” and “most likely.” One cause of the Civil War was most likely for this reason: Which is NOT the first step in simplifying a fraction?

MULTIPLE-CHOICE There are typically 4 to 5 choice items for each stem; with 3 choices for younger students. The written length of all the choices should be very similar. (They should be shorter than the length of the stem.)

MULTIPLE-CHOICE Randomize the sequence of correct letter choices/answers. Make sure that all 4 letter choices are equally used throughout the test. Avoid using an “ALL OF THE ABOVE” choice, but you can use “NONE OF THE ABOVE” which is more difficult.

The next to highest level of Bloom’s Taxonomy is which of the following? A. Comprehension B. Analysis C. Synthesis D. None of the above

MULTIPLE-CHOICE Questions are written specifically to assess the objectives that are meant to be tested. There is a fairly equal number of items asked about each one of the objectives. All of the options seem realistic; none are ‘silly’, unreasonable, or clearly wrong.

PRACTICE on Multiple Choice items

(Selected Response) MATCHING ITEMS Focuses on important concepts, and measures test objectives. Two parallel lists of words or phrases requiring students to match entries on one list with entries on the other list.

MATCHING Use lists that are homogeneous or in the same kind of group. Lists should be brief, with the shorter words or phrases (CHOICES) listed on the right. Add extra responses, more than is actually needed, to the list on the right. (2-3 more) Alphabetize or arrange the responses on the right in a chronological sequence or numerical order from least to greatest.

Describe the directions clearly: •How the matches are to be made; •How many times the responses can be used. Finally, place the matching items on the same page.

NOTE how specific and clear the directions are: Directions: On the line to the left of each military conflict listed in Column A, write the letter of the U.S. president in Column B who was in office when that military conflict was concluded. Each name in Column B may be used no more than once. There are 3 extra choices in Column B that will not be used. Column B A. Bush B. Clinton C. Eisenhower D. Johnson E. Nixon F. Roosevelt G. Truman H. Wilson Column A ____1. World War I ____2. World War II ____3. Korea ____4. Vietnam ____5. First Persian Gulf

PRACTICE on MATCHING items

LIST A LIST B Hurricane 1. Spawns in tropical waters in the north Pacific Typhoon 2. Torrential rains in the rain forest C. Cyclone 3. Spawns in tropical waters in the western Pacific Monsoon 4. Forms in the Atlantic and northern Pacific Has this MATCHING SET been written correctly?

LIST A LIST B Hurricane 1. Spawns in tropical waters in the north Pacific Typhoon 2. Torrential rains in the rain forest C. Cyclone 3. Spawns in tropical waters in the western Pacific Monsoon 4. Forms in the Atlantic and northern Pacific •There are NO DIRECTIONS • LIST A should actually become List B on the right side; •The choices moved to the right side should be alphabetized; • There needs to be 2 or 3 more choices than definitions.

Directions: On the line to the left of each statement, write the letter of the philosopher from LIST B on the right with the descriptions in LIST A. Use each philosopher only once. You will not use two choices in List B. LIST A LIST B 1. ___Thought government was a A. Hobbes science requiring experts. B. Marx 2.___ Described methods of achieving political unity. C. Machiavelli 3. ___Founded Communism. D. Durkheim 4. ___Thought that human nature E. Plato made absolute monarchy inevitable. F. Socrates G. Aristotle Is this MATCHING SET well-written?

1.___ Thought government was a A. Hobbes science requiring experts. Directions: On the line to the left of each statement, write the letter of the philosopher from LIST B on the right with the descriptions in LIST A. Use each philosopher only once. You will not use two choices in List B. LIST A LIST B 1.___ Thought government was a A. Hobbes science requiring experts. B. Marx 2.___ Described methods of achieving political unity. C. Machiavelli 3. ___ Founded Communism. D. Durkheim 4. ____Thought that human nature E. Plato made absolute monarchy inevitable. F. Socrates G. Aristotle •The choices in List B on the right side should be alphabetized. • DIRECTIONS error – 3 not used

2. ___ The highest humanistic need. C. Love & Belonging Directions: On the line to the left of each statement, write the letter of the humanistic needs from LIST B with the descriptions of LIST A. Use each humanistic need only once. You will not use two choices in List B. LIST A LIST B ___Student benefits from praise A. Egocentrism and recognition of accomplishments. B. Esteem 2. ___ The highest humanistic need. C. Love & Belonging 3.____ Need for food, water, air, etc. D. Physiological 4. ____ Student needs to have friends. E. Safety 5. ____ Student is free from bullying. F. Self-Actualization F. Zone of Proximal Is this MATCHING SET okay?

___Student benefits from praise A. Egocentrism Directions: On the line to the left of each statement, write the letter of the humanistic needs from LIST B with the descriptions of LIST A. Use each humanistic need only once. You will not use two choices in List B. LIST A LIST B ___Student benefits from praise A. Egocentrism and recognition of accomplishments. B. Esteem 2. ___ The highest humanistic need. C. Love & Belonging 3.____ Need for food, water, air, etc. D. Physiological 4. ____ Student needs to have friends. E. Safety 5. ____ Student is free from bullying. F. Self-Actualization F. Zone of Proximal Technically, not all of LIST B choices are humanistic needs from Maslow.

CONSTRUCTED RESPONSES Students are “constructing” or forming their own responses without the help of provided choices. These type of responses often use higher levels of thinking.

CONSTRUCTED-RESPONSES Completion Items A FILL-IN-THE-BLANK: A DIRECT QUESTION: …. with an exact answer or number that the student provides. The teacher should expect one specific and brief response by the way the item is written.

Fill-in-the-blank or Direct Question SHORT-ANSWER : Fill-in-the-blank or Direct Question The first President of the United States was ____________. Who was the first President of the United States? Word the item carefully to get the exact brief response you want… otherwise you might get unexpected answers that are true. The first President of the United States was a man . The NAME OF the first President of the United States was ________________.

FILL-IN-THE-BLANK If you use incomplete statements, position the answer blanks at the end of the question or towards the right side. Keep the answer blanks the same length so you are not giving away hints (longer blanks means longer answers; shorter blanks mean shorter words,etc.) Every odd year in Lindsborg, a Swedish festival is held in October that is called ___________.

FILL-IN-THE-BLANK For incomplete statements, use only one or two blanks per sentence. Don’t “swiss-cheese” the item. If you include a “word bank” of answer choices, make sure it is on the same page or facing page, not on the backside. Every ____ year in Lindsborg, a ____________festival is held in ________________ that is ___________ and is called ______________________________. NO ! A Swedish fall festival is held in the month of _________ every odd year, and this festival is called __________.

SHORT-ANSWER: Direct Question Direct Questions are useful for one correct, very short answer, such as specific vocabulary, people, places, numbers, etc. What is the vocabulary word for ruling families in ancient China? What number comes next in this sequence: 2, 4, 8, 16, 32 ? Where is the Amazon River? (CAN BE ANSWERED CORRECTLY WITH DIFFERENT ANSWERS) Word the question carefully to get the intended response.

PRACTICE on COMPLETION ITEMS (Fill-in-the-Blank) CONSTRUCTED RESPONSES

A typhoon is called a ____________ in the Atlantic and Northern Pacific regions. Approximately ____ people have been pronounced dead as a result of this super typhoon. Haiyan is the Chinese word for __________, a type of _____ that lives over the __________ and returns to ____________ only for breeding. Are these Completion Items well-written?

A typhoon is called a ____________ in the Atlantic and Northern Pacific regions. Approximately ____ people have been pronounced dead as a result of this super typhoon. Haiyan is the Chinese word for __________, a type of _____ that lives over the __________ and returns to ____________ only for breeding. •The blanks are different sizes suggesting the answer is a longer or shorter word. • The answer blanks should all be at the end of sentences. •Too many “swiss-cheese” blanks in the third sentence.

CONSTRUCTED RESPONSES OTHER EXAMPLES OF CONSTRUCTED RESPONSES

Bloom's level? KNOWLEDGE "What" CONSTRUCTED-RESPONSE In this Math constructed-response, the student gives a short numerical or verbal response. A small number of items require students to fill in a form according to written instructions. Bloom's level? KNOWLEDGE "What"

Bloom's level? APPLICATION "Use", KNOWLEDGE "Tell" CONSTRUCTED-RESPONSE In Social Studies, this constructed-response makes use of a map, with written directions on how the student is to find the information, and give the correct one word response. Bloom's level? APPLICATION "Use", KNOWLEDGE "Tell" Use the map of South America to answer this question. Tell the name of the country labeled with an “A” on this map? Answer: ______________________ A

Bloom's level? ________ "Identify," "Write" CONSTRUCTED-RESPONSE In Language Arts, this constructed-response gives written directions for the student to find information from a sample sentence and give the correct phrase. The red balloon floated into the sky. Read the underlined sentence, above. Identify the prepositional phrase in this sentence and write it in the answer blank. Answer: ______________________ Bloom's level? ________ "Identify," "Write"

CONSTRUCTED-RESPONSE Higher level thinking A SHORT ANSWER requiring a short sentence or two to further explain a definition, key points, or examples. A (short or long) ESSAY with specific and clear questions and directions guiding the student’s extended response to several key points described in the essay prompt.

SHORT-ANSWER Pay attention to Bloom’s Taxonomy verbs as you form questions for higher-level thinking responses. Each question can ask for a couple of responses in the answer such as descriptions, comparisons, examples, etc.

SHORT-ANSWER Constructed Response Short QUESTION-ANSWER items require students to provide a wider variety of possible answers, and may be in the combination of numbers, words, phrases, drawings, or sentences. Students may also be asked to refer to a text, sentence, diagram, graph/chart, equation, problem, or respond by using a drawing or graphic organizer along with writing. (i.e. Explain what…..?….means on the chart. Give 2 examples or conclusions you draw from the chart.)

SHORT-ANSWER Bloom's level? KNOWLEDGE, COMPREHENSION, In Physical Education, this constructed- response requires the student to explain in several sentences. It is actually a four-part response. Notice the higher level of cognitive thinking this requires. Name two kinds of overhead serves in volleyball. Explain the benefits of using each one. Answer: (Two kinds of overhead serves are the top spin and the floater. Top spin would be used for speed projection and the floater for height. ) SCORING ON 4 POINTS AND/OR USING A RUBRIC Bloom's level? KNOWLEDGE, COMPREHENSION, APPLICATION and ANALYSIS

QUESTION-ANSWER Bloom's level? KNOWLEDGE, COMPREHENSION, In Language Arts, this constructed-response requires the student to recall knowledge, give an opinion, and refer to a poetic text to explain the information required. It is also a four-part response. Bloom's level? KNOWLEDGE, COMPREHENSION, APPLICATION and ANALYSIS QUESTION: How does the word "person" give you a clue as to the meaning of personification? Why do you think a writer would want to use personification in a poem? Identify and write two examples of personification found in the poem at the left. The sky is low THE sky is low, the clouds are mean, A traveling flake of snow Across a barn or through a rut Debates if it will go. A narrow wind complains all day How some one treated him; Nature, like us, is sometimes caught Without her diadem. Emily Dickinson

ESSAYS Higher level of cognitive abilities used: students synthesize, evaluate and compose in writing. May be more difficult for the student to write and express their knowledge. May be more difficult for the teacher to reliably score student responses, unless they prepare the question with the specific answers in mind.

GUIDELINES FOR ESSAYS Describe clearly to students their task, with a writing prompt. Explain how extensive their responses should be.(Use complete sentences, write a paragraph, no less than 30 words, 5 sentences, half a page, etc.) Be specific about the point value for particular items and how many examples, key points they are to give. (A definition for each and 3 examples, the purpose and your opinion for each, worth 10 points, etc.)

GUIDELINES FOR ESSAYS Mentally ‘preview’ likely student responses and put yourself in their minds as you create essay questions. Consider the time it will take for students to complete their essays. Limit the number of essay questions to 2 or 3 on the whole test and make them purposeful.

SCORING ESSAYS Score responses holistically and/or analytically. Create a tentative scoring key, and even write out the responses that you are expecting. Score responses holistically and/or analytically. Score one essay item at a time on all of your students’ tests, before scoring the next item.

SCORING ESSAYS Decide if you want to count off for errors or incomplete sentences and how many points. You may only want to score by the content or correct answer. Either way, be clear in the directions what you are expecting for the student to gain all the points.

PRACTICE on Short Responses and Essays CONSTRUCTED-RESPONSE

•LOOK FOR BLOOM’S LEVELS of lower to higher levels of thinking required. •HOW MANY POINTS SHOULD EACH BE WORTH? •Does the question clearly ask for a specific number of examples or things to write about for each answer? Give some characteristics of a typhoon. 2. List some details of the devastation of the typhoon. 3. Describe the damage and the effect the typhoon had caused to the Philippines.

•LOOK FOR BLOOM’S LEVELS of lower to higher levels of thinking required. •HOW MANY POINTS SHOULD EACH BE WORTH? •Does the question clearly ask for a specific number of examples or things to write about for each answer? 4. Describe the differences and similarities of a hurricane and a typhoon. 5. Describe the characteristics of a hurricane, typhoon and cyclone.

•LOOK FOR BLOOM’S LEVELS of lower to higher levels of thinking required. •HOW MANY POINTS SHOULD EACH BE WORTH? •Does the question clearly ask for a specific number of examples or things to write about for each answer? 6. Write down your thoughts over this event, and how we could help those in need. 7. In about two paragraphs, describe the effects typhoon Haiyan had on the Philippines.

•LOOK FOR BLOOM’S LEVELS of lower to higher levels of thinking required. •HOW MANY POINTS SHOULD EACH BE WORTH? •Does the question clearly ask for a specific number of examples or things to write about for each answer? In two or three paragraphs, describe what a typhoon is, and what it would be like to live through one. (If you can imagine). Do this from knowledge gained from the articles given to you.