1. An Analysis of NYS Regents Physics Exam Issues J. Zawicki Science Education, BSC STANYS DAL, Physics Mentor

2. 9/25/03 NYS Regents Physics -- Zawicki 2 Additional Researchers Michael Jabot, SUNY Fredonia Kathleen Falconer, SUNY BSC Dan MacIsaac, SUNY BSC David Henry, SUNY BSC Rita Fischer, Elba Central School

3. 9/25/03 NYS Regents Physics -- Zawicki 3 Presentation Overview Issues from the field Exam findings Issues and events Future directions…

4. 9/25/03 NYS Regents Physics -- Zawicki 4 Issues From the Field What are issue(s) are uppermost in your mind at this time?

5. 9/25/03 NYS Regents Physics -- Zawicki 5 Exam Findings Call for student papers, STANYS, NYS Section AAPT >2000 student papers were submitted Results from across NYS

6. 9/25/03 NYS Regents Physics -- Zawicki 6 Initial Distribution of Papers

7. 9/25/03 NYS Regents Physics -- Zawicki 7 Rasch Analysis Parts A and B-1; B-2 and C presented together Concepts – Difficulty – how well did the students do? – Discrimination – how well did an item distinguish between high and low performing students? – Rasch transformation – log transformation

8. 9/25/03 NYS Regents Physics -- Zawicki 8

9. 9/25/03 NYS Regents Physics -- Zawicki 9 “Easy” Items – A, B-1 ItemKeyR1R2R3R4%CNDE Q45357268601589.8958-2.17 Q40340380710684.2956-1.69 Q362122777382181.1958-1.46 Q383125475913179.2956-1.35 Q32129749384278.2958-1.28

10. 9/25/03 NYS Regents Physics -- Zawicki 10

11. 9/25/03 NYS Regents Physics -- Zawicki 11

12. 9/25/03 NYS Regents Physics -- Zawicki 12

13. 9/25/03 NYS Regents Physics -- Zawicki 13

14. 9/25/03 NYS Regents Physics -- Zawicki 14 “Difficult” Items – A, B-1 ItemKeyR1R2R3R4%CNDE Q151324134257633.89550.67 Q113697635245836.79550.54 Q443132023738740.49570.39 Q729539432014841.19570.36 Q3441223714641843.69570.25

15. 9/25/03 NYS Regents Physics -- Zawicki 15

16. 9/25/03 NYS Regents Physics -- Zawicki 16

17. 9/25/03 NYS Regents Physics -- Zawicki 17

18. 9/25/03 NYS Regents Physics -- Zawicki 18

19. 9/25/03 NYS Regents Physics -- Zawicki 19 “Easy Items” B-2, C ItemKeyR0R1R2%CNDE Q59116458--96.6474-3.35 Q542275139889.0476-2.84 Q65127447--94.3474-2.81 Q58134440--92.8474-2.56 Q692236039191.0474-2.32

20. 9/25/03 NYS Regents Physics -- Zawicki 20

21. 9/25/03 NYS Regents Physics -- Zawicki 21

22. 9/25/03 NYS Regents Physics -- Zawicki 22

23. 9/25/03 NYS Regents Physics -- Zawicki 23

24. 9/25/03 NYS Regents Physics -- Zawicki 24 “Difficult” Items B-2, C ItemKeyR0R1R2%CNDE Q621266207--43.84730.25 Q481253223--46.84760.13 Q531246230--48.34760.07 Q741243231--48.74740.05 Q561242233--49.14750.04

25. 9/25/03 NYS Regents Physics -- Zawicki 25

26. 9/25/03 NYS Regents Physics -- Zawicki 26

27. 9/25/03 NYS Regents Physics -- Zawicki 27

28. 9/25/03 NYS Regents Physics -- Zawicki 28 Reading Level McLaughlin-SMOG Additional Analysis Pending

29. 9/25/03 NYS Regents Physics -- Zawicki 29 Initial Analysis, Reading Levels Exam AdministrationGrade Equivalent June 20008 th June 200110 th (Low) June 200210 th (High) June 200311 th

30. 9/25/03 NYS Regents Physics -- Zawicki 30 Conceptual Level Modification of Bloom – Knowing – Using 1 – Using 2 – Integrating

31. 9/25/03 NYS Regents Physics -- Zawicki 31 Exam Conceptual Level Exam AdministrationConceptual Level June 20001.61 June 20011.59 June 20021.89 June 20031.74

32. 9/25/03 NYS Regents Physics -- Zawicki 32 Exam Format ExamAdministrationMCWP June 200075113 January 200175103 June 200175114 January 200275113 June 2002452412 August 200247218 January 2003502710 June 2003472916

33. 9/25/03 NYS Regents Physics -- Zawicki 33 AP Correlation Approximately 0.69 Precise value varied from year to year

34. 9/25/03 NYS Regents Physics -- Zawicki 34 NYS Physics/AP B

35. 9/25/03 NYS Regents Physics -- Zawicki 35 Listserv Comments Latency -- Incomplete/unfinished tests Power Tests – assessments where time does not play a significant role in the score

36. 9/25/03 NYS Regents Physics -- Zawicki 36 Issues Impact of Scaled Scoring on student scores in the standards-based exams Direction of future exams

37. 9/25/03 NYS Regents Physics -- Zawicki 37 Sessions NYSUT Session with Commissioner Mills Science Education Consortium – August Regents Session – September Senate Hearings NYS AAPT Meeting (SUNY Binghamton) Physics Summit (Albany Area)

38. 9/25/03 NYS Regents Physics -- Zawicki 38 Additional Directions? Where do we go from here?