1. Physics Concept Surveys TDSB physics team May 28, 2004

2. Problem Students do not learn physics concepts very well under traditional classroom instruction

3. Problem Students do not learn physics concepts very well under traditional classroom instruction They memorize “plug and chug” techniques, get 80% on a physics exam, but only score 50% on a multiple choice concept survey

4. Problem Students do not learn physics concepts very well under traditional classroom instruction They memorize “plug and chug” techniques, get 80% on a physics exam, but only score 50% on a multiple choice concept survey In fact, when concept surveys are applied pre-post, gains are only.286 (30% to 50%)

5. Purpose 2003-2004: develop concept surveys for Ontario SPH 3U teachers to use as pre-post diagnostics to measure conceptual gains

6. Purpose 2003-2004: develop concept surveys for Ontario SPH 3U teachers to use as pre-post diagnostics to measure conceptual gains 2004-2005: encourage teachers to try new interactive teaching strategies to see if they lead to increased conceptual gains

7. History of Project I August, 2003 – team of physics teachers developed a plan: Shawn Bullock, Peter Carlow, Doug Hayhoe, Jason Law

8. History of Project I August, 2003 – team of physics teachers developed a plan: Shawn Bullock, Peter Carlow, Doug Hayhoe, Jason Law Sept. 2003 – we studied existing concept surveys, such as Force Concept Inventory, as well as the Ontario curriculum

9. History of Project I August, 2003 – team of physics teachers developed a plan: Shawn Bullock, Peter Carlow, Doug Hayhoe, Jason Law Sept. 2003 – we studied existing concept surveys, such as Force Concept Inventory, as well as the Ontario curriculum Oct. to Dec. 2003 – drafted and revised five surveys of approx. 20 MC questions each (Each item is referenced to the curriculum)

10. Force and Motion: Unit 1 Survey 12. A large truck collides head on with a small compact car. During the collision … A. The truck exerts a greater amount of force on the car than the car exerts on the truck B. The car exerts a greater amount of force on the truck than the truck exerts on the car C. Neither exerts a force on each other. The car gets smashed because it is in the way D. The truck exerts a force on the car but the car does not exert a force on the truck E. The truck exerts the same amount of force on the car as the car exerts on the truck

11. Curriculum References for Unit 1, #12 Specific Curricular Expectation  state Newton’s laws, and apply them to explain the motion of objects in a variety of contexts

12. Curriculum References for Unit 1, #12 Specific Curricular Expectation  state Newton’s laws, and apply them to explain the motion of objects in a variety of contexts Achievement Chart Criteria  demonstrate understanding of concepts, laws, and theories

13. Light and Geometric Optics: Unit 4 17. In the following diagram, two different lenses with different focal lengths are placed in sequence in an optical device. An object is located where the arrow is in the diagram. Left Right The second image, after light from the object has passed through both lenses, would appear to be: A. to the left of the converging lens and right side up B. between the two lenses and right side up C. between the two lenses and upside down D. to the right of the diverging lens and right side up E. to the right of the diverging lens and upside down

14. Curriculum References for Unit 4, #17 Specific Curricular Expectation  describe and explain, with the aid of light- ray diagrams, the characteristics and positions of the images formed by lenses

15. Curriculum References for Unit 4, #17 Specific Curricular Expectation  describe and explain, with the aid of light- ray diagrams, the characteristics and positions of the images formed by lenses Achievement Chart Criteria  understand relationships among concepts

16. History of Project II January to May, 2004 – Field-tested concept surveys in 20 schools across the TDSB

17. History of Project II January to May, 2004 – Field-tested concept surveys in 20 schools across the TDSB April to June, 2004 – conducted item analysis of questions on concept surveys

18. History of Project II January to May, 2004 – Field-tested concept surveys in 20 schools across the TDSB April to June, 2004 – conducted item analysis of questions on concept surveys May 7, 2004 – physics team met to examine each question on surveys and recommend for inclusion, revision or deletion (including Dianne Ness and Tasha Richardson)

19. Field-test Results Overall Unit 1: 9 schools, 306 students, 47% average

20. Field-test Results Overall Unit 1: 9 schools, 306 students, 47% average (4 schools, 127 students, pre-post 35%-52%)

21. Field-test Results Overall Unit 1: 9 schools, 306 students, 47% average (4 schools, 127 students, pre-post 35%-52%) Unit 2: 9 schools, 364 students, 52% average

22. Field-test Results Overall Unit 1: 9 schools, 306 students, 47% average (4 schools, 127 students, pre-post 35%-52%) Unit 2: 9 schools, 364 students, 52% average (3 schools, 69 students, pre-post 41%-52%)

23. Field-test Results Overall Unit 1: 9 schools, 306 students, 47% average (4 schools, 127 students, pre-post 35%-52%) Unit 2: 9 schools, 364 students, 52% average (3 schools, 69 students, pre-post 41%-52%) Unit 3: 7 schools, 262 students, 49% average

24. Field-test Results Overall Unit 1: 9 schools, 306 students, 47% average (4 schools, 127 students, pre-post 35%-52%) Unit 2: 9 schools, 364 students, 52% average (3 schools, 69 students, pre-post 41%-52%) Unit 3: 7 schools, 262 students, 49% average (5 schools, 170 students, pre-post 33%-49%)

25. Field-test Results Overall Unit 1: 9 schools, 306 students, 47% average (4 schools, 127 students, pre-post 35%-52%) Unit 2: 9 schools, 364 students, 52% average (3 schools, 69 students, pre-post 41%-52%) Unit 3: 7 schools, 262 students, 49% average (5 schools, 170 students, pre-post 33%-49%) Units 4 & 5: results are still being collected

26. Force and Motion: Unit 1 Survey 12. A large truck collides head on with a small compact car. During the collision … A. The truck exerts a greater amount of force on the car than the car exerts on the truck B. The car exerts a greater amount of force on the truck than the truck exerts on the car C. Neither exerts a force on each other. The car gets smashed because it is in the way D. The truck exerts a force on the car but the car does not exert a force on the truck E. The truck exerts the same amount of force on the car as the car exerts on the truck

27. Item Assessment Criteria Difficulty level – What % of students answered the question correctly?

28. Item Assessment Criteria Difficulty level – What % of students answered the question correctly? Discrimination coefficient – Did a higher % of better students answer correctly?

29. Item Assessment Criteria Difficulty level – What % of students answered the question correctly? Discrimination coefficient – Did a higher % of better students answer correctly? Pre-post gain – Did more students answer correctly after studying the topic?

30. Difficulty Level – 51% Correct 12. A large truck collides head on with a small compact car. During the collision … Students writing post-test306 A. The truck exerts a greater amount of force on the car than the car exerts on the truck135 B.The car exerts a greater amount of force on the truck than the truck exerts on the car 6 C.Neither exerts a force on each other. The car gets smashed because it is in the way 1 D. The truck exerts a force on the car but the car does not exert a force on the truck 8 E.The truck exerts the same amount of force on the car as the car exerts on the truck156

31. Discrimination Coefficient = 0.31 12. A large truck collides head on with a small compact car. During the collision … 306 students divided into higher 153 & lower 153 A.The truck exerts a greater amount of force on the car than the car exerts on the truck 45 90 B.The car exerts a greater amount of force on the truck than the truck exerts on the car 3 3 C.Neither exerts a force on each other. The car gets smashed because it is in the way 0 1 D.The truck exerts a force on the car but the car does not exert a force on the truck 3 5 E.The truck exerts the same amount of force on the car as the car exerts on the truck 102 54

32. Pre-post Gain = 0.54 (21% to 64%) 12. A large truck collides head on with a small compact car. During the collision … Students writing both pre and post 127 Pre Post A.The truck exerts a greater amount of force on the car than the car exerts on the truck 79 41 B.The car exerts a greater amount of force on the truck than the truck exerts on the car 5 1 C.Neither exerts a force on each other. The car gets smashed because it is in the way 3 0 D.The truck exerts a force on the car but the car does not exert a force on the truck 8 4 E.The truck exerts the same amount of force on the car as the car exerts on the truck 27 81

33. Field-test Results for Unit 3, #14 14. Consider the following wave properties: 1. frequency 2. wavelength 3. Amplitude Which of these properties listed above never changes as a wave travels along one rope? A.1 only B.2 only C.3 only D.1 and 2 only E.1 and 3 only

34. Difficulty Level – 21% Correct 14. Consider the following wave properties: 1. frequency 2. wavelength 3. Amplitude Students writing post-test258 Which of these properties listed above never changes as a wave travels along one rope? A.1 only 91 B.2 only 47 C.3 only 48 D.1 and 2 only 55 E.1 and 3 only 17

35. Discrimination Coefficient = 0.19 14. Consider the following wave properties: 1. frequency 2. wavelength 3. Amplitude 258 students divided into higher 129 & lower 129 Which of these properties listed above never changes as a wave travels along one rope? A.1 only57 34 B.2 only14 33 C.3 only 14 34 D.1 and 2 only40 15 E.1 and 3 only 5 12

36. Pre-post Gain = 0 (21.7% to 21.1%) 14. Consider the following wave properties: 1. frequency 2. wavelength 3. amplitude Students writing both pre and post 166pre post Which of these properties listed above never changes as a wave travels along one rope? A.1 only 40 58 B.2 only35 29 C.3 only 45 34 D.1 and 2 only36 35 E.1 and 3 only 12 10

37. Future of Project I June to August, 2004 – preparation of “final” surveys for pre-post use in 2004-2005

38. Future of Project I June to August, 2004 – preparation of “final” surveys for pre-post use in 2004-2005 They will have this format Part 1. Units 1-2, Mechanics, 35 items Part 2. Units 3-5, Waves, Light, EM, 35 items

39. Future of Project II Sept-Jan, 2005: collection of baseline data (volunteer teachers will use revised concept surveys to collect pre-post course data without changing their teaching strategies)

40. Future of Project II Sept-Jan, 2005: collection of baseline data (volunteer teachers will use revised concept surveys to collect pre-post course data without changing their teaching strategies) Feb-June 2005: collection of experimental data (the same teachers will collect pre-post course data on a second course, in which they use substantially different teaching strategies)