Faculty Research & Creativity Forum October 30, 2008 D. MacIsaac, Physics J. Zawicki, Earth Sciences and Science Education L. Gomez, Physics SUNY Buffalo State College T. Johnson, Erie 1 BOCES, WNYRIC

False Dichotomies2 Frameworks Syllabi Guides Blueprints Benchmarks Objective tests Performance assessments Portfolios Teacher Observations Group Activities Program Evaluations Curriculum Standards Assessment/Evaluation SystemInstructional Program alignment validity correlation Instructional styles Print materials Equipment Facilities Technology Community

 Conventional ◦ Difficulty (Facility) ◦ Discrimination ◦ Response pattern  Rasch  Item format  Difficulties analyzed in the context of issues: 1 ◦ Student ◦ Instructional (Teacher, School) ◦ Testing 1 NYS Biology Mentor Network False Dichotomies3

4 StudentInstruction Testing Validity You didn’t teach the associated core major understandings. You didn’t reinforce the core understandings enough. You taught the core content wrong Content Knowledge? Literacy/Reading Comprehension? Question interpretation Skills? Misconception? –From previous instruction? –From culture contexts? –Insufficient reinforcement? Effort? Difficulty (Facility) Level? Discrimination? Placement on exam? Visual distraction by nearby (graphic) items? Style of Question? Flawed item?

Students do not “see” action/reaction pairs. Modified Benjamin Bloom: knowing using integrating False Dichotomies5

10 Earth’s mass is approximately 81 times the mass of the Moon. If Earth exerts a gravitational force of magnitude F on the Moon, the magnitude of the gravitational force of the Moon on Earth is: (1) F (3) 9F (2) F/81(4) 81F False Dichotomies6 ItemDifficulty1234NR

 Students have a difficult time differentiating between mass and weight. False Dichotomies7

A 2.00 kilogram object weighs 19.6-newtons on Earth. If the acceleration due to gravity on Mars is 3.71 meters per second 2, what is the object’s mass on Mars? 1) 2.64 Kg 2) 2.00 Kg 3) 19.6 N 4) 7.42 N False Dichotomies8 ItemDifficulty1234NR

A horizontal force of 8.0 newtons is used to pull a 20.-newton wooden box moving toward the right along a horizontal, wood surface, as shown. 61. Starting at point P on the diagram in your answer booklet, use a metric ruler and a scale of 1.0 cm = 4.0 N to draw a vector representing the normal force acting on the box. Label the vector F N. [1] D = Calculate the magnitude of the fricitional force acting on the box. (Show all work.) [2] D = Determine the magnitude of the net force acting on the box. [1] D = Determine the mass of the box. [1]D = Calculate the magnitude of the acceleration of the box. [Show all work.] [2]D = 0.80 False Dichotomies9

 Sample items  Range of item difficulties… False Dichotomies10

False Dichotomies11

False Dichotomies12

False Dichotomies13

False Dichotomies14

False Dichotomies15

False Dichotomies16

Teachers often underestimate the critical role played by conceptual physics reasoning which is quite difficult to teach and learn, and focus exclusively on numeric problem solving which is often already taught to students in other classes – like in chemistry and mathematics courses. The conceptual questions on NYSED Regents’ Physics exams are routinely amongst the most difficult items for HS physics students, and a common reaction from students and less successful teachers is that these are “trick questions.” However, these conceptual questions lie at the heart of the physics, and calculating physical quantities that cannot be reliably interpreted by students shows both a weakness in physics teaching and physics learning. Calculation emerges from, is validated and interpreted by conceptual understanding, and conceptual learning is not useful in physics unless it can be used to numerically predict natural phenomena.  Additional information: J. Zawicki False Dichotomies

We have observed amongst pre-service and new HS Physics Teachers a common belief that physics problems that do not involve mathematical calculations (aka “conceptual problems”) are easier than those involving mathematical calculations (aka “numeric problems”). Teachers are very likely to allocate student instruction and learning resources in physics according upon their beliefs. NY HS Physics teachers are driven by the NYSED Regents’ Physics Exam. Hence, we have selected and analyzed several elementary mechanics questions from about 1500 student scores on various NYSED Regents Physics Exams that challenge teacher perceptions of difficulty and numeric vs conceptual content. False Dichotomies18

The June 2003 Regent’s Physics exams juxtaposed several numeric and conceptual problems including (on the same page) items #12 and #15. Item #15, a conceptual problem requiring simple definition (of inertia or mass, part of the definition of momentum) that presented extraneous numeric data was later determined to be the most difficult item on that exam offering – only about thirty eight percent of about a thousand students were able to answer it correctly. Item #12, a standard “cookbook-like” problem applying these same definitions and principles (calculating and conserving momentum in a collision) via multiple simple calculations turned out to be in the easiest third of the exam that year, answered correctly by over seventy percent of the students that year. Example 1: Momentum, inertia and mass. False Dichotomies19

A significant fraction of the NYSED Regents physics exam items make use of mass and weight calculations. Students have asked to calculate weight (a force) from mass and apply this information in different problems on every recent exam, often in complex multi-step problems as shown in the example. Students do relatively well on these items. Periodically the exam directly challenges the student to differentiate between mass and weight. These items are usually the most difficult items on that offering of the exam.  Example 2: Mass and weight. False Dichotomies20

NYSED Regents Physics exams prevalently include calculation of inverse square law forces (electrostatic and gravitation), often combined with proportional reasoning and often requiring interpretations of graphic representations of these forces. When presented as a direct calculational exercise as seen on item 14 from the June 2007 exam, students perform well, despite the fact that this calculation requires significant expertise with scientific notation and calculators. When asked for simple restatements of these gravitational interactions (the definition of any force), students find these questions about twice as difficult, amongst the most difficult on the exam. See item 12 from the June 2005 exam. Example 3: Gravitation interactions. False Dichotomies21