Assessing Assessment

Ideas For Varying Test Format Information about a student’s thinking and understanding can be obtained by modifying many multiple choice (MC) items and adding a second part in which the student is asked to “explain” or “defend” his/her response.

Example of a Middle school two-part item on the topic of expansion and contraction 1a. When heat energy is added to matter, the matter A. Expands B. Contracts C. Freezes D. Condenses 1b. Explain your selection in 1a.

A Typical Multiple Choice Item A.Steam turns into water B.Water turns into steam C.Water turns into ice D.Ice turns into water

Item Analysis 1.Faulty stem. It would be more accurate to say: “An example of a solid turning into a liquid is when” (D. Ice turns into water). 2.Faulty science: Ice just doesn’t simply “turn into water.” The item totally ignores the role of (heat) energy during a change of state. Ice (a solid) turns into a liquid (water) when sufficient energy is added to break the forces holding the particles (water molecules) in a (crystalline) lattice. 3.Most importantly, the student selection reveals very little about what the student does or does not understand about change of states of matter. The item is testing nothing more than a definition—OK for a language arts test, but not very good for a science test.

1a. When heat energy is added to matter, the matter A. Expands B. Contracts C. Freezes D. Condenses 1b. Explain your selection in 1a. Either choices A or B can be correct with proper explanation (all materials other than solid water (ice) at 4 degrees Celsius do expand when heat is added. The two-part item can then be scored on a scale which reflects the student’s awareness of the general behavior of matter and the uniqueness of water.

1a. When heat energy is added to matter, the matter A. Expands B. Contracts C. Freezes D. Condenses 1b. Explain your selection in 1a. Full credit: Student marks A or B, or A & B and explains the general rule governing matter and the unique relationship for ice at 4 degrees Celsius. Partial credit: Student marks A or B but explains only the general rule or the special case with ice at 4 degrees Celsius, but not both. Partial credit: Student marks A or B but gives no reasonable explanation of the general rule or specific rules.

A Better Multiple Choice Item *What happens when a solid is heated and changes into a liquid? A.Particles in the solid get lighter and are able to float. B.The solid gets softer until it completely melts. C.The solid gets warmer and warmer until it melts. D.Particles in the solid break away from each other

A Better Multiple Choice Item A.Particles in the solid get lighter and are able to float. B.The solid gets softer until it completely melts. C.The solid gets warmer and warmer until it melts. D.Particles in the solid break away from each other 1.The stem is clearer and scientifically more accurate.

A Better Multiple Choice Item A.Particles in the solid get lighter and are able to float. B.The solid gets softer until it completely melts. C.The solid gets warmer and warmer until it melts. D.Particles in the solid break away from each other 2. The best answer is consistent with the molecular model of matter and conceptually appropriate for a middle elementary level student.

A Better Multiple Choice Item A.Particles in the solid get lighter and are able to float. B.The solid gets softer until it completely melts. C.The solid gets warmer and warmer until it melts. D.Particles in the solid break away from each other 3. The incorrect answers are drawn from classroom & research data on ideas that students hold about changes of states of matter & provide information on which a teacher can plan further activities.

Some Activities  Measuring the mass of an ice cube & the water after the ice has melted is a test of selection ‘A’ A.Particles in the solid get lighter and are able to float.

Some Activities  Measuring the temperature of an ice/water mix as the ice melts will reveal that the temperature doesn’t change during the melting.

Multiple Choice Analysis Conventional wisdom about the validity of test items:  Item difficulty  Item difficulty: the percentage of students selecting the correct response. If everyone gets the item wrong (too difficult), or everyone gets the item correct (too easy) the item should be thrown out.  Item reliability  Item reliability: the correlation between item score and total score. Students with the highest overall score should get the item correct; students with lowest overall score should get the item wrong.

Constructed Response Items  Consists of short or extended response items.  There is one general question or proposition, and the student is asked to respond in writing.  This type of assessment is very powerful -- it allows the students to express themselves and demonstrate their reasoning related to a topic.  CR questions often demand the use of higher level thinking skills, such as analysis, synthesis, and evaluation.

Things to Consider  Specify how the students should respond: Sea water contains dissolved salts and is not considered potable. Describe a procedure that can be used to obtain a cup of drinking water from a bucket of sea water. Clearly indicate the method used to separate the water from salt and collect the pure water.

Things to Consider  Provide information about the value/weight of the question and how it will be scored: Sea water contains dissolved salts and is not considered potable. Describe a procedure that can be used to obtain a cup of drinking water from a bucket of sea water (20 Points). Clearly indicate the method used to separate the water from salt and collect the pure water

Things to Consider  Demand high order thinking skills: Sea water contains dissolved salts and is not considered potable. Describe a procedure that can be used to obtain a cup of drinking water from a bucket of sea water. List 2 possible solutions

Final Thoughts  Don’t add charts, graphics, etc if they aren’t needed.  Don’t ask students to do too much  Remember you are assessing their knowledge, not trying to kill them