Classroom Assessment: Concepts and Applications Chapter 6: Creating Achievement Tests Classroom Assessment: Concepts and Applications

In Chapter 6 We Will Study: Selection and Supply Test Items Higher-Level Questions Guidelines for Writing and Critiquing Test Items

Achievement Test The most commonly used procedure for gathering formal evidence about student learning.

A good assessment plan takes many things into consideration. 1st: identifying important instructional objectives 2nd: selecting question formats that match the objectives 3rd: deciding whether to construct one’s own test or use one from a textbook 4th: providing good instruction 5th: providing a review and information about the test

What can go wrong??? Test items can be poorly constructed Test can be objectively scored The result? Students don’t have a fair chance to show what they have learned Information is not provided for valid decision making

Tests are composed of short communications called questions or items. Test items must be … Brief Set clear problems for students to think about Complete and independent of other questions Be stated in clear, precise language Linked to the educational objectives

Selection and Supply Test Items Please turn to page 146. 1/12 Selection and Supply Test Items Please turn to page 146. Selection Items Selection items are those in which a student selects a correct answer from among a number of options presented. Selection Items Include: 1. Multiple choice 2. True-False 3. Matching

Selection: Multiple-Choice Items 2/12 Selection: Multiple-Choice Items These consist of a stem, which presents the problem or question to the student, and A set of options, or choices, from which the student selects an answer. May be used to assess higher-level thinking, but primarily used to assess factual knowledge and comprehension levels. Stem Options

Multiple Choice: Advantages/Disadvantages 3/12 Multiple Choice: Advantages/Disadvantages Advantages: Easy to score Several items can be completed by students in a short period of time Disadvantages: Doesn’t allow students to construct, organize, and present their own answers Susceptible to guessing

Selection: True-False Items 4/12 Selection: True-False Items Requires students to classify a statement into one of two categories, such as true or false; yes or no. Primarily used to assess factual knowledge and comprehension. May be used to assess higher-level thinking. T F

True-False: Advantages/Disadvantages 5/12 True-False: Advantages/Disadvantages Advantages: Easy to score Several items can be completed by students in a short period of time Disadvantages: Doesn’t allow students to construct, organize, and present their own answers Susceptibility to guessing

Selection: Matching Items 6/12 Selection: Matching Items Consist of a column of premises, a column of responses, and directions for matching the two. The same set of options or responses is used for all the premises.

Matching: Advantages/Disadvantages 7/12 Matching: Advantages/Disadvantages Advantages: Easy to score Decreases the amount of reading students must perform in order to display knowledge of several terms, people, or facts Disadvantages: Limited mainly to assessing lower-level behaviors

Supply Items Please turn to page 148. 8/12 Supply Items Please turn to page 148. Supply items are those in which the student supplies or constructs his or her own answer. Includes: Short Answer Completion (Fill-in-the-blank) Essay

Short-Answer & Completion Items 9/12 Short-Answer & Completion Items Short answer: Presents the problem with a direct question. What is the name of the first president of the United States? Completion: Presents the problem as an incomplete sentence with blanks to fill in. The name of the first president of the United States is __________. In each case, the student must supply his or her own answer, typically a word, phrase, number, or sentence.

Short Answer & Completion: Advantages/Disadvantages 10/12 Short Answer & Completion: Advantages/Disadvantages Advantages: Fairly easy to construct Diminishes the likelihood that students will guess answers Disadvantages: Tends to mainly assess factual knowledge or comprehension.

Essay Items: Advantages 11/12 Essay Items: Advantages Essay questions give students the greatest opportunity to construct their own responses. Most useful for testing higher-level thinking skills such as analyzing, synthesizing, and evaluating. Also the primary way teachers assess students’ ability to organize, express, and defend ideas.

Essay Items: Disadvantages 12/12 Essay Items: Disadvantages Time consuming to answer and score. Permit testing only of a limited amount of students’ learning. Tend to favor students with good writing ability.

Higher-Level Questions Growing emphasis on teaching and assessing students’ higher-level thinking Solving problems, interpreting charts, explaining something in one’s own words, identifying relationships, and carrying out specific activities

Interpretive Exercises A common form of multiple-choice item that can assess higher-level thinking. Gives students some information or data and then asks students to interpret, comprehend, analyze, apply, or synthesize it. Graphs, charts, reading passages, pictures, or tables.

Interpretive Exercises: Disadvantages Cannot show how students organize their ideas when solving a problem. Difficult to construct Heavy reliance on students’ reading ability

Interpretive Exercises: Should meet 5 general guidelines 3/8 A. Relevance: The exercise should be related to the instruction provided to students. If it is not, it should not be used. B. Similarity: The material presented in the exercise should be new to the students, but similar to material presented during instruction. C. Brevity: There should be sufficient information for students to answer the questions, but the exercises should not become tests of reading speed and accuracy. D. Answers not provided: The correct answers should not be found directly in the material presented. Interpretation, application, analysis, and comprehension should be needed to determine correct answers. E. Multiple questions: Each interpretive exercise should include more than one question to make most efficient use of time.

Guidelines for Writing & Critiquing Test Items To improve tests, test items should… 1. Cover important objectives emphasized in instruction 2. Be written clearly and simply: Seven Rules 3. Be reviewed for misleading statements, confusing formatting, or excess verbiage before testing

Rule 1: Write Clearly & Simply: Avoid ambiguous and confusing wording and structure. Students are prevented from figuring out what they are being asked and can’t demonstrate their learning.

Rule 2: Write Clearly & Simply: Use appropriate vocabulary. If students can’t understand the vocabulary used in test questions, their test scores will reflect their vocabulary deficiencies rather than how much they learned from instruction.

Rule 3: Write Clearly & Simply: Keep questions short and to the point. Test items should quickly focus students on the question being asked.

Rule 4: Write Clearly & Simply: Page 162 Write items that have one correct answer. With the exception of essay questions, most paper-and-pencil test items are designed to have students select or supply one best answer.

Rule 5: Write Clearly & Simply: Give information about the nature of the desired answer. Essay questions should focus students’ answers on the major points covered by instruction. Inform students about the nature and scope of the expected answer.

Rule 6: Write Clearly & Simply: Page 164 Do not provide clues to the correct answer. Test item writers should take care not to provide grammatical clues, implausible option clues, or specific determiner clues.

Rule 7: Write Clearly & Simply Don’t overcomplicate. Avoid using numbers or words that overcomplicate a given problem. This increases the likelihood of errors and does not accurately assess the students’ ability.

Review Items Before Testing Reread test items. Match each item to an objective. Modify or delete items that don’t link to a lesson objective. Ask a colleague, etc. to review the items critically. Edit test based on feedback.

2. Minor differences among organisms of the same kind are known as Each item has at least one fault. Read each item, identify the fault(s) in it and rewrite the item to correct the fault. Robert Fulton, who was born in Scotland and came to the US in 1843, is best known for his invention of the steamboat that he called the Tom Thumb. True False Rule 1: Confusing wording 2. Minor differences among organisms of the same kind are known as Heredity Variations Adaptation Natural selection Rule 6: Clue provided

4. F. Scott Fitzgerald wrote _________________________. Each item has at least one fault. Read each item, identify the fault(s) in it and rewrite the item to correct the fault. 3. The recall of factual information can best be assessed with a __________ item. a.matching b. objective c. essay d. short-answer Rule 6: Clue Provided 4. F. Scott Fitzgerald wrote _________________________. Rule 4: Many correct answers.

Each item has at least one fault Each item has at least one fault. Read each item, identify the fault(s) in it and rewrite the item to correct the fault. 5. Although the experimental research completed, particularly that by Hansmocker, must be considered too equivocal and the assumptions viewed as too restrictive, most testing experts would recommend that the easiest method of significantly improving paper-and-pencil achievement test reliability would be to a. increase the size of the group b. increase the weighting of the items c. increase the number of items d. increase the amount of testing time. Rule 1: Confusing; Rule : Inappropriate vocabulary; Rule 3: Not kept short and to the point Rule 7: Over complicated

6. Boston is the most important city in the Northeast. T F Each item has at least one fault. Read each item, identify the fault(s) in it and rewrite the item to correct the fault. 6. Boston is the most important city in the Northeast. T F Rule 1: Confusing wording (undefined: most important) 7. An electric transformer can be used a. for storing up electricity. b. to increase the voltage of alternating current. (correct answer) c. it converts electrical energy into direct current. d. alternating current is changed to direct current. Rule 1: Confusing sentence structure

Chapter OLC Review Visit Chapter 6 of the text website for chapter quizzes, related websites, and other helpful study materials. www.mhhe.com/airasian6e

Chapter 6: Creating Achievement Tests Name_________________ What are the differences between selection and supply items? What are the advantages and disadvantages of selection items? What are the advantages and disadvantages of supply items? 4. What are the differences between higher- and lower-level test items? 5. What are examples of clues to be avoided in multiple-choice, true-false, completion, and matching items? 6. What is an interpretive exercise and why is it a useful method for assessing higher-level thinking? 7. Three guidelines for constructing test questions are: Cover important topics Write clearly and simply Review items before testing. How does each of these guidelines lead to improved test questions?

Review of Chapter 6: The Frayer Model The Frayer Model (Frayer, Frederick, and Klaumeier, 1969) helps students learn the meanings of key concepts through word categorization. Using this method, students form an understanding of concepts by learning the attributes, choosing examples, and choosing non-examples of the concept. Students can use the Frayer Model before, during, or after reading to learn more about a topic. Utilizing this instructional model extends student comprehension of the topic and aids in retention of the information. 38

Definition Characteristics Examples Non-examples Word

Definition Characteristics Examples Non-examples

Multiple-Choice Items True-False Items Short-Answer Items Form 6 Pairs Pairs count off Construct a Frayer Model for your assigned concept. Multiple-Choice Items True-False Items Short-Answer Items Completion Items Essay Items Interpretive Exercises

Social Studies Journal Pocket Social Studies Journal