What are ‘Constructed-Response’ test questions? How can I write excellent ‘constructed response’ test questions? How well do ‘constructed response’ test items measure student learning? What are the strengths and weakness of ‘constructed response’ test items?
Trace the history of constructed-response assessment. Explain how essay items can differ in terms of purpose and level of complexity. Compare and contrast restricted-response and extended-response essay items. Describe the principles involved with developing effective essay items. Develop effective essay items for a given content area.
Discuss the strengths and weakness of essay items. Describe the principles involved with grading essays. Demonstrate the ability to grade essay in a reliable and valid manner. Describe the principles involved with developing effective short-answer items. Develop effective short-answer items for a given content area.
Discuss the strengths and weaknesses of short-answer items. Discuss prominent issues to be considered when deciding whether to use selected- response or constructed-response items.
1) Selected Response: requires a student to select a response from a list of alternatives. Ex: multiple-choice, true-false, and matching items. 2) Constructed Response: requires students to create or construct a response. Ex: Essay and short-answer.
Greek Testing Format (as far back as 400 B.C): Oral testing. Adopted by Romans and continued when universities were established during the dark ages. Oral exams persist in Western universities (e.g., master’s thesis or doctoral dissertation defense). Issues w/ Oral Exams: think time, student ability to present ideas orally, validity and reliability, scoring, teacher time.
1) What is the relationship between your essay items and your actual classroom instruction? 2) What is the relationship between your ressay items and your instructional objectives. 3) Do you want a restricted-response answer from students or an extended-response answer?
4) Do you want to ask a series of interrelated essay questions or completely different questions? 5) How many essay questions do you want to give on an assessment? 6) Do you want to give optional essay items?
Content: testing for cognitive achievement. Purpose is to determine what student knows or can produce. All levels of Bloom’s can be measured. Essay testing in ‘content’ context should not penalize student deficient in skills unrelated to the content being assessed. Ex: poor organization, misspelling)
Style: often used in writing composition classes. All objectives are related to organization, structure, phrasing, transition, etc. Grammar is not assessed in ‘style’ context.
Grammar: Used in writing composition. Purpose to examine student’s ability to apply grammatical rules and writing mechanics (e.g., spelling, punctuation, etc.). Content and style are not scored. Combination: Can combine style-content, content-grammar, or style-grammar. Be clear on how essays will be graded.
Essay items can be written to measure objectives at all levels of the cognitive taxonomy. Practice: Write an essay question for each level of Bloom’s Taxonomy, in your content area.
Do you want to give open-ended questions that allow a large degree of choice by students in answering the questions? Do you want to restrict the responses of students by asking very specific questions that require very specific answers? Pro/Con: Open-ended questions can be difficult to grade reliably. Restricted-response questions can limit creativity and unique student responses.
Extended: (usually a minimum of one page in length) Describe all the factors listen in your textbook and in the classroom instruction about the decision to use the atomic bomb during World War II. Restricted: In two or three paragraphs describe three major factors involved in the decision to use the atomic bomb during World War II. Which factor is the most important? List this factor and provide a rationale for why it is the most important.
Includes a series of essay questions about a specific piece of information. Allows an in-depth probe of the topic. Can be combined with other assessment formats such as short answer, fill in blank, sentence completion. (would include a map) 1) In terms of transportation, where are the two best possible locations? Provide a detailed rationale of why there are good locations for transportation. 2) In terms of access to natural resources, which two sites are best? List at least two reasons why these sites have the best access to natural resources.
What exactly do you want to assess? Do you want to focus on just content when you grade essays? Do you want to grade on just style? Do you want to strike a balance between grading for content knowledge vs. grading on stylistic aspects? What kind of scoring system do you want to use? Do you want to use a numerical scoring system or some other type of system?
1) Holistic Scoring: Ex: The following areas will be used to determine your overall grade on this essay test. 2) Analytical Scoring: Ex: Rate each category on a 4-point scale. 3) Stylistic Scoring: Ex: The following will be rated on a four point scale: Introduction, clarity, grammar, syntax, punctuation, etc. 4) Content Scoring: Ex: The following will be worth 8 pts. each: thesis statement, citations, etc.
1) Purpose, goal, or end in view: broad definition of purpose. 2) Question at issue or problem to be solved: specific issues or problems. 3) Point of view or frame of reference: various perspectives on an issue. 4) Reasoning dimensions: scientific reasoning, articulating key concepts, relevancy vs. irrelevancy of data and concepts. 5) Assumptions and implications: background and consequents of central problem. 6) Conclusions: rational, logical, relevant solution, or future considerations.
Rubrics are only as good as your ability to communicate your grading expectations. Students need to have an understanding up front of how they are going to be graded on essays, presentations, products, etc. Use student feedback to help you with ‘assessment’ review. Look critically at test ‘items’, essay questions after used with students. Focus group: student feedback on specific projects, processes, etc.
Fill-in-the-blank items: Ex: Missing letter, word, etc. Cloze test items: leaves out words or a series of words in a paragraph or phrase. Ex: ‘Don’t Forget the Lyrics’. Short –answer: students supply one word of a couple of words in response to a question. Ex: What is the name of an arboreal marsupial that is native to North America? Oral Exams: interviews, series of questions, sometimes from a team of interviewers. Ex: Doctoral Defense.
See checklist, p. 240 Strengths: Require recall, not just recognition. Well suited for quantitative problems and interpretation of graphic material. Short answers allow greater sampling of content area than essay questions. Relatively easy to write Weaknesses: Scoring, low ‘Bloom’s levels.
Grading: making sure grader is ‘fair’. Examine yourself for potential biases. Ex: The way it ‘looks’, length, student name, etc. Rubrics: well-constructed, give enough information for students to understand requirements. Individual Test Items: continue to examine test questions for clarity, bias, quality answers, etc. How do CR Assessments fit in your overall grading policies?