David McConnell Marine, Earth, and Atmospheric Sciences and Office of Faculty Development NCSU Clicker Pedagogy in Larger Classes: How Using Clickers Can Improve Learning Office of Faculty Development

Seven Principles of Good Practice in Undergraduate Education 1.Encourages student-faculty contact 2.Develops cooperation among students 3.Encourages active learning 4.Provides prompt feedback 5.Emphasizes time on task 6.Communicates high expectations 7.Respects diverse talents and ways of learning Chickering & Gamson, AAHE Bulletin, 1987, p. 3-7 Can clickers promote these practices?

When you ask a question in class, what proportion of your students will raise their hand or otherwise indicate that they have an answer? A. More than 50% B. About 30% C. About 10% D. 0-10%

Instructor grading of short answer and essay questions On-going assessment through student dialog in small classes Computer grading of multiple choice questions using bubble- sheets Understanding Student Learning Learning assessment systems More instructor understanding of learning Less instructor understanding of learning

Clicker Pedagogy: Peer Instruction & Conceptests It’s the message, not the medium Clickers in Effective Pedagogy

Harvard students in a traditional calculus-based introductory physics class scored an average of 70% on the pre-test. Predict the average post-test score. A. 72% B. 78% C. 84% D. 90% Crouch, C.H., Mazur, E., 2001, American Journal of Physics, v. 69, #9, p

What we know about student learning 1.Students learn key concepts better when they have opportunities to actively monitor their understanding. 2.Knowledge is socially constructed and people learn best in supportive social settings when working with peers. 3.Students become better learners when we challenge them to answer questions that require the use of higher order thinking skills.

Peer instruction (& Conceptests) Development of technique by Eric Mazur, Harvard Short lecture (10-20 minutes) Conceptest – conceptual multiple choice question Individual students signal answers Student groups may discuss answers (peer instruction) Explanation of correct answer Mazur, E., 1997, Peer instruction: A user’s manual: Prentice Hall, 253p. Pedagogy for Feedback Devices U. Of Massachusetts model skips individual student answers

Examine the map and answer the question that follows. How many plates are present? Example: Earth Science Conceptest McConnell, D.A., et al., 2006, Journal of Geoscience Education, v. 54, #1, p Results when using physical models: (56%; 84%) Geology conceptest database : interactive/conctest.html a. 3 b. 4 c. 5 d. 6 a. 3 (26%; 0%) b. 4 (19%; 18%) c. 5 (44%; 75%) d. 6 (11%; 7%) Individual responses Post-discussion responses

Student Performance on Conceptests About a third of questions in an Earth Science class were asked twice. An average of 45% of students responded correctly on the first attempt and 63% answered correctly after peer instruction. Five questions had fewer correct answers after peer instruction.

Range of student scores for a semester of conceptests Student Performance on Conceptests Nobody has ever averaged above mid- 80s for semester Most students miss at least a third of questions asked

Crouch, C.H., Mazur, E., 2001, American Journal of Physics, v. 69, #9, p FCI pretest score FCI score gain on post-test Traditional Class Peer Instruction Classes n = Began PI Refined conceptests Changed text Open ended reading questions Mazur’s results Impact of Alternative Pedagogy

Dori, Y.J. and J. Belcher, J Journal of the Learning Sciences 14(2). Impact of Alternative Pedagogy Poulis et al., (1998) Results from 5,000+ physics students – increase in pass rates (55%  80%) in classes that used audience paced feedback (clickers + student discussion) Dori & Belcher (2004) Compared pre- and post-test scores for traditional and technology-enabled physics classes – learning gains were greater (27% vs. 52%) for technology enhanced class Poulis, C., Massen, C., Robens, E., & Gilbert, M American Journal of Physics, v.66 #5, p

Clicker Pedagogy: Why this works It’s the message, not the medium Clickers in Effective Pedagogy

Students completed a short reading assignment. Population A studied the passage twice (7 minutes each time). Population B studied the passage once and then took a recall test. Two days later, both groups were tested on their recall of information. Predict the result. A.Population A scored higher on the test. B.Population B scored higher on the test. C.There was no difference in test score. Roediger & Karpicke, 2006, Perspectives in Psychological Science, v. 1, p

Test Enhanced Learning Roediger & Karpicke, 2006, Perspectives in Psychological Science, v. 1, p students complete a reading assignment (~250 words) Population A studied the passage twice (7 minutes each time) Population B studied the passage once and then took a test Both populations then tested at 5 minute, 2 day, and 1 week intervals ABABAB Population B retained more knowledge after 2 days, 1 week

Peer Learning Assessment Smith et al., 2009, Science, v. 323, January 2, p U. Of Colorado Genetics Course Conceptest Responses Individual responses Peer instruction responses Follow up individual responses Conceptest posed All CorrectCorrect after peer instruction

The Value of Peer Instruction Experimental Group: Students took physics test individually, then again as a pair. Control Group: Students took test individually. Singh, C., American Journal of Physics, v.73 #5, p Proportion of pairs of students who both got the question wrong on the first test but correct on “paired” test: 29% Mean score on second exam for experimental group: 74% Mean score on second exam for control group: 64% Students in both groups answered similar questions on a second exam two weeks later.

The Value of Peer Instruction Students taught key concepts using one of four methods. Student learning assessed by proportion of correct answers to open ended questions on same concepts on final exam Crouch, C.H., Fagen, A.P., Callan, J.P., & Mazur, E., American Journal of Physics, v.72 #6, p No demonstration Observation of demonstration w/explanation Prediction prior to demo with a conceptest Prediction prior to demonstration using discussion & a later conceptest % correct answers 61 70* 77* 82* Teaching method n = ; * = statistically significant result vs. no demonstration

Importance of Student Reflection Dunning et al., Current directions in psychological science, v.12 #3, p The weakest students often do not realize that they do not understand key concepts Doubly cursed: Students who can’t answer questions correctly can’t self- diagnose their lack of ability

The Value of Peer Reflection Experimental Group: Three 2-minute pauses per lecture, student discussion of lecture content with peer. Control Group: No pauses for discussion in lecture. Ruhl, Hughes, and Schloss., Teacher Education and Special Education, v.10 #1, p All students completed a free recall exercise at end of lecture and delayed multiple choice test 12 days later. Exp. Group – mean number of facts recalled : 22.97* Cont. Group – mean number of facts recalled : Exp. Group – MC test average score : 84.39* Cont. Group – MC test average score : * statistically significant gain

Value of Attendance Moore et al., 2003, American Biology Teacher, v. 5, p Emphasized attendance, showed data graph weekly Average attendance 70% Average grade 73% Verbal encouragement to attend (less emphasis) Average attendance 59% Average grade 64% Science classes at University of Minnesota

Clicker Pedagogy:... and it makes me feel good It’s the message, not the medium Clickers in Effective Pedagogy

Instructor Satisfaction Survey (n = 35)

Student Satisfaction vs. Class Level n = 1597

Impact on Students Total Students Surveyed: 1327 PedagogyTechnologyOtherTotal What do you like best about the use of conceptests and clickers? # of Responses % of Responses 86.8%8.5%4.7% 100% What do you like least about the use of conceptests and clickers? # of Responses % of Responses 19.1%73.3%7.6% 100% Student Comment Matrix

Benefits of Technology & Pedagogy What did you like best about the use of the conceptests and CPS? “That it really helped me participate more in class.” Natural Science Biology “CPS forced me to review class materials. This helped to reinforce my memory/knowledge.” Microbiology “I was able to gage my knowledge level to others in class.” Emergency Management. “It makes you want to learn more and enjoy the class more.” Basic Mathematics II “I knew what I had to study.” Human Diversity “I actually had to figure out problems to answer them, so I understood it better.” Principles of Chemistry “Not having to talk to participate.” Government and Politics

Drawbacks of Technology & Pedagogy What did you like least about the use of the conceptests and CPS? “We got into discussion groups, but didn’t discuss.” Human Diversity “Questions sometimes are a little difficult to understand.” Criminal Case Management “The stress of missing questions.” Statistics “It takes time during class that we could be using to take notes.” Human Diversity “Some questions didn’t give enough time.” Macroeconomics “Attendance and having to come to class all of the time.” Government and Politics “Don’t make questions all at the end of lecture. Throw some in the middle of the lecture.” Macroeconomics

Suggestions for Using Clickers Some suggestions from our experience: Use the clickers everyday to insure students bring them with them to class and value their use Make questions sufficiently challenging to them worth asking – aim for correct response rates 50-70% Avoid grading headaches – low stakes assessment, consider using for participation points Bigger is best – use in lower level, gen. ed. classes Less value if class already uses active pedagogy strategies or if students already participate fully

Seven Principles of Good Practice 1.Encourages student-faculty contact 2.Develops cooperation among students 3.Encourages active learning 4.Provides prompt feedback 5.Emphasizes time on task 6.Communicates high expectations 7.Respects diverse talents and ways of learning Chickering & Gamson, AAHE Bulletin, 1987, p. 3-7

Clicker Pedagogy: * Writing Good Questions * It’s the message, not the medium Clickers in Effective Pedagogy

Teaching and learning goals can be ordered using Bloom’s Taxonomy Knowledge Comprehension Application Analysis Synthesis Evaluation memorization and recall understanding using knowledge taking apart information reorganizing information making judgements Assessment with Clickers Degrees of correctness Right/Wrong answers More complex questions call for more sophisticated guides and responses

Introductory Exercise Examine the six questions on page 2 of the handout. Assume you are a student in classes where these questions would be appropriate. Rank the questions from the easiest to most challenging based on the character of the question and nature of knowledge needed to answer it correctly.

Classify using Bloom ’ s Taxonomy A.Which one of the following values approximates best to the volume of a sphere with radius 5m? a) 2000m³ b) 1000m³ c) 500m³ d) 250m³ e) 125m³ B.How successful were recent income tax cuts in spurring economic growth? C.What is the capital of Maine? D.How would you restructure the school day to reflect children’s developmental needs? E.Contrast the floor of the Atlantic Ocean with the shape of a bathtub. F.Which statements in the President’s State of the Union address were based on facts and which were based on assumptions?

Which is a synthesis question? A.A B.B C.C D.D E.E F.F

Which is an application question? A.A B.B C.C D.D E.E F.F

Bloom ’ s Taxonomy Knowledge Which one of the following persons is the author of "Das Kapital"? a) Mannheim b) Marx c) Weber d) Engels e) Michels Comprehension Fill in the blank to complete the analogy. The yolk is to the egg as the ____________ is to Earth. a) crustb) mantlec) core d) asthenosphere

Bloom ’ s Taxonomy Comprehension In the landscape below, how would the amount of rainfall change at location X if the mountain eroded down to the dashed line? a. Rainfall would increase b. Rainfall would decrease c. Rainfall would stay the same

Bloom ’ s Taxonomy Analysis Read carefully through the paragraph below, and decide which of the options a, b, c, or d is correct. Rising saturated air undergoes: i) adiabatic cooling as air contracts due to decreasing pressure with increasing altitude; and, ii) warming due to the latent heat of condensation as water vapor is converted to liquid water droplets. a. The word “contracts” should be replaced by “expands”. b. The word “condensation” should be replaced by “evaporation”. c. The word “warming” should be replaced by “cooling”. d. The word “altitude” should be replaced by “elevation”.

Bloom ’ s Taxonomy a)The assertion and the reason are both correct, and the reason is valid. b)The assertion and the reason are both correct, but the reason is invalid. c)The assertion is correct but the reason is incorrect. d)The assertion is incorrect but the reason is correct. e)Both the assertion and the reason are incorrect. Evaluation Judge the sentence in italics according to the criteria given below: "The United States took part in the Gulf War against Iraq BECAUSE of the lack of civil liberties imposed on the Kurds by Saddam Hussein's regime.“

Any Questions? Clickers and Effective Pedagogy