1. Workshop on Peer Instruction: Making Good Clicker Questions Beth Simon Computer Science and Engineering University of California, San Diego Credits: Sara Harris, Earth and Ocean Sciences, UBC Clicker Resource Guide from CU-SEI and CWSEI Rosie Redfield (for clicker question contribution) Roland Stull (for clicker question contribution)

2. By the end of today’s talk you should be able to… LG1: Describe your reasons for wanting to use PI and how that informs what you should plan to do LG2: Identify your concerns about using PI and present pros and cons associated with it LG3: Refer back to the Clicker Resource Guide when creating clicker questions LG4: Analyze a clicker question for it’s Bloom’s taxonomy level

3. What are your primary concerns in your current course? 1. 2. 3. 4. 5.

4. Which of those do you think Peer Instruction will help? Discuss

5. What concerns do you have about using Peer Instruction? 1. 2. 3. 4. 5.

6. End-of-Term survey, upper-level science course (200 students) University of British Columbia What do students think? What do you think worked well in this course, and why? # Responses “clicker questions helped me understand major concepts and how to apply the material” “it gave me a sense of what I understood and what I didn't.” “clicker questions were really helpful because they made me to think about the material during class.” “I liked the clicker questions because they made me think deeper into particular concepts, but I did find them very challenging.”

7. Stages of clicker use 1. Stage 1 – asking simple, primarily factual questions. 2. Stage 2 – asking more challenging conceptual questions, or questions where the answer is not obvious and critical points could be argued. --Perhaps after short mini-lectures (10 min) 3. Stage 3 – Lecture is structured around a set of challenging clicker questions that largely embody the material students are to learn.

8. More Detail: Clicker Resource Guide Executive Summary Overview the rest –So you know what you can return to

9. Types of Clicker Questions What are you going to ask? What do you want them to know?

10. Bloom’s taxonomy VERBS for Stage 1 (knowledge) Know Define Memorize Repeat Record List Recall Name Relate

11. Level 1 example Who is/was the 12 th president of UBC? A. Martha Piper B. David Strangway C. Leonard S. Klinck D. Frank F. Wesbrook E. Stephen Toope

12. Bloom’s taxonomy VERBS for Stage 2 (Comprehension, Application) Translate Interpret Apply Employ Use Demonstrate Dramatize Practice Illustrate Operate Sketch Discuss Describe Recognize Explain Express Identify Locate Report Review Tell Restate

13. B > b > b l B– black bb or bb' brown b'b' cinnamon Two black cats are crossed. Which of the following outcomes is not possible? A.Only black kittens. B.Black and brown kittens. C.Black and cinnamon kittens. D.Black, brown and cinnamon kittens. E.All are possible. Level 2 example

14. Bloom’s taxonomy VERBS for Stage 3 (Analysis, Synthesis, Evaluation) Compose Plan Propose Design Formulate Assemble Construct Collect Create Design Organize Distinguish Analyze Differentiate Appraise Calculate Compare Contrast Diagram Debate Relate Examine Judge Appraise Evaluate Rate Value Revise Score Select Choose Estimate Measure

15. Level 3 example

16. How to write a clicker question: 1. Pick a concept or skill you think is important and/or difficult for students. 2. If you know them, identify any common misconceptions to use as distractors. 3. Write the question as simply and clearly as possible. There’s lots of literature about writing effective MC questions. It’s not easy. Here’s a link to get started with the issues around MCQs: http://teambasedlearning.apsc.ubc.ca/?page_id=163http://teambasedlearning.apsc.ubc.ca/?page_id=163

17. How to respond to histograms: What would you do if you got this histogram? A. B. C. D. E. BCDEA

18. How to respond to histograms: Example: Why do we have seasons on Earth? A.Because the Earth is spinning B.Because the Earth is tilted C.Because the Earth is spherical D.Because the Earth is sometimes closer, sometimes farther from the Sun E.Because the Earth’s equator is closer to the Sun than the poles

19. Chemistry: Equilibrium 64. (Equilibrium, Le Châtelier's principle; Ch. 5 & 9 "Companion") The body-centered cubic (bcc) phase of a metal has a 68% packing efficiency, while the face-centered cubic (fcc) phase of the same metal has a 74% packing efficiency. For the equilibrium bcc metal = fcc metal as pressure increases, the equilibrium A)shifts to the left B)shifts to the right C)is unaffected

20. Chemistry: Kinetics and Mechanism 129. (Catalysts) Which curve illustrates the effect of a catalyst on the reaction diagram, given that it speeds up the rate of a reaction? A B C D

21. How many times is each set of code executed? Pixel[] pixelArray = this.getPixels(); int value = 0; Pixel p = null; for(int index = 0; index < pixelArray.length; index++); { p = pixelArray[index]; value = p.getRed(); value = (int) (value * 0.5); p.setRed(value); } 1)Solo: (30 sec) 2)Discuss: (1min) 3)Group: (30 sec)

22. How many times is each set of code executed? Pixel[] pixelArray = this.getPixels(); int value = 0; Pixel p = null; for(int index = 0; index < pixelArray.length; index++); { p = pixelArray[index]; value = p.getRed(); value = (int) (value * 0.5); p.setRed(value); } 1)Solo: (30 sec) 2)Discuss: (1min) 3)Group: (30 sec)

23. What picture most accurately describes what this code does ? Pixel[] pixelArray = this.getPixels(); int value = 0; Pixel p = null; for(int index = 0; index < pixelArray.length; index++) { p = pixelArray[index]; q = pixelArray[index+1]; p.setRed(q.getRed()); p.setBlue(q.getRed()); p.setGreen(q.getGreen()); } 1)Solo: (30 sec) 2)Discuss: (2min) 3)Group: (30 sec)

24. What picture most accurately describes what this code does ? Pixel[] pixelArray = this.getPixels(); int value = 0; Pixel p = null; for(int index = 0; index < pixelArray.length; index++) { p = pixelArray[index]; q = pixelArray[index+1]; p.setRed(q.getRed()); p.setBlue(q.getRed()); p.setGreen(q.getGreen()); } 1)Solo: (30 sec) 2)Discuss: (2min) 3)Group: (30 sec)

25. A method in Picture.java… Pixel p; for (int foo = 0; foo < getWidth(); foo++) { for (int bar 0; bar < getHeight(); bar++) { p = getPixel(foo, bar); p.setColor(Color.BLACK); } 1)Solo: (30 sec) 2)Discuss: (2min) 3)Group: (30 sec) Nested Loops: How do they work? What order are pixels changed?

26. A method in Picture.java… Pixel p; for (int foo = 0; foo < getWidth(); foo++) { for (int bar 0; bar < getHeight(); bar++) { p = getPixel(foo, bar); p.setColor(Color.BLACK); } 1)Solo: (30 sec) 2)Discuss: (2min) 3)Group: (30 sec) Nested Loops: How do they work? What order are pixels changed?

27. Why does this code have an error? A.It tries to access pixelArray[-1] B.It tries to access pixelArray[0] C.It tries to access pixelArray[pixelArray.length] D.It tries to access pixelArray[pixelArray.length+1] E.None of the above 1)Solo: (30 sec) 2)Discuss: (2min) 3)Group: (30 sec) Pixel[] pixelArray = this.getPixels(); int value = 0; Pixel p = null; for(int index = 0; index < pixelArray.length; index++) { p = pixelArray[index]; q = pixelArray[index+1]; p.setRed(q.getRed()); p.setBlue(q.getRed()); p.setGreen(q.getGreen()); }

28. Why does this code have an error? A.It tries to access pixelArray[-1] B.It tries to access pixelArray[0] C.It tries to access pixelArray[pixelArray.length] D.It tries to access pixelArray[pixelArray.length+1] E.None of the above 1)Solo: (30 sec) 2)Discuss: (2min) 3)Group: (30 sec) Pixel[] pixelArray = this.getPixels(); int value = 0; Pixel p = null; for(int index = 0; index < pixelArray.length; index++) { p = pixelArray[index]; q = pixelArray[index+1]; p.setRed(q.getRed()); p.setBlue(q.getRed()); p.setGreen(q.getGreen()); }

29. DEBUGGING: Where is the best place to put a print statement? Pixel[] pixelArray = this.getPixels(); int value = 0; int index = 0; while (index < pixelArray.length) { value = pixelArray[index].getRed(); value = (int) (value * 0.5); pixelArray[index].setRed(value); index = index + 1; } 1)Solo: (30 sec) 2)Discuss: (1min) 3)Group: (30 sec)

30. DEBUGGING: Where is the best place to put a print statement? Pixel[] pixelArray = this.getPixels(); int value = 0; int index = 0; while (index < pixelArray.length) { value = pixelArray[index].getRed(); value = (int) (value * 0.5); pixelArray[index].setRed(value); index = index + 1; } 1)Solo: (30 sec) 2)Discuss: (1min) 3)Group: (30 sec)

31. SPEEDY: What does that code do? A.Makes it higher pitched B.Makes it louder C.Makes if lower pitched D.Makes it quieter E.Makes a silent sound public void funky() { SoundSample[] noiseArray = this.getSamples(); int zzz = 0; for (int i=0;i<noiseArray.length) { SoundSample sample = noiseArray[i]; int foo = sample.getValue(); zzz += foo; } int yyy = zzz / noiseArray.length; for (int i = 0; i < noiseArray.length; i++) { SoundSample sample = noiseArray[i]; sample.setValue(yyy); } 1)Solo: (20 sec) 2)Discuss: (1 min) 3)Group: (20 sec)

32. SPEEDY: What does that code do? A.Makes it higher pitched B.Makes it louder C.Makes if lower pitched D.Makes it quieter E.Makes a silent sound public void funky() { SoundSample[] noiseArray = this.getSamples(); int zzz = 0; for (int i=0;i<noiseArray.length) { SoundSample sample = noiseArray[i]; int foo = sample.getValue(); zzz += foo; } int yyy = zzz / noiseArray.length; for (int i = 0; i < noiseArray.length; i++) { SoundSample sample = noiseArray[i]; sample.setValue(yyy); } 1)Solo: (20 sec) 2)Discuss: (1 min) 3)Group: (20 sec)

33. Best Practice Quick Sheet Give reading quizzes before each lecture Foster group discussion –Assign groups (and keep the same ones all term) –Require them to reach consensus and vote “the same” –(Rotate ownership of discussion within groups) Ask challenging questions –That make for meaningful discussion –35-75% solo -> >75% group Respond to student experience –Ask them to provide explanations first (then you can re-word) –If the group vote correctness is low Take time to review, explain, clarify Frequently (once a week, then often) remind students of the value of Peer Instruction