1. Copyright John Broida, This work is the intellectual property of the author. Permission is granted for this material to be shared for non-commercial, educational purposes, provided that this copyright statement appears on the reproduced materials and notice is given that the copying is by permission of the author. To disseminate otherwise or to republish requires written permission from the author.

2. John Broida University of Southern Maine
Adding Quizzing and Clickers to Introductory Psychology Helped Students
John Broida
University of Southern Maine

3. In 1999, our introductory course was very difficult for students
1000 students per year took the class
This is a large enrollment course on a campus with 10,000 students
Required of majors in 6 departments
Met the distributed education requirement in social science
This means that many were not psychology majors
25% were Dropping, Failing or Withdrawing (DFW)

4. Teaching was done by Ph. D. psychologists.
None had a strong background in teaching.
The instructors knew what they were talking about, but had not necessarily thought about what it was to be a good teacher.
The course was typically lecture with three exams.
No feedback on progress until the exam.
Some instructors provided extra credit for attendance to some lectures.
Exams were based entirely on the textbook.

5. One instructor was quizzing students weekly
Using the overhead projector to pose 2 multiple choice questions each week.
Students turned in pieces of paper with answers on them.
Took a lot of time to grade, and then enter the grades into the grade book.
Students in his classes tended to do better on exams than those in other classes.
Was it the instructor or the method? Where his tests easier than those of others teaching the course?

6. That instructor started to use disk-based and server-based quizzes.
Using Question Mark to allow students to answer 20 questions per chapter.
Better representation of the chapter than 2 questions
Quizzes could be taken repeatedly, using randomized selection of items from a large pool.
Recorded the highest grade
Exam grades were noticeably better from what had been the case previously.
Same book, same instructor, same test items.
Drawback was in recording grades into gradebook

7. Things got a bit better with web-based quizzes.
Multiple choice questions
Students could take the quizzes repeatedly, given random selection of items.
Using Bravo! as the course management system.
Much easier for students who did not want to take quizzes on campus.
Instructor did not have to enter grades into a spreadsheet.
It takes the same amount of instructor time for 50 students as it does for 300!

8. Based on this experience, the department received a grant to do more
From what was to become the National Center for Academic Transformation.
The goals were to reduce costs associated with instruction in large enrollment classes.
Costs interpreted as cost per student to successfully deliver the course.
We achieved significant cost savings by using the same materials.
We wanted to reduce our DFW rate.

9. After several missteps, we developed something that actually worked.
Weekly web-based quizzes.
Covered material not yet discussed in lecture.
Students were better prepared for class.
Random selection of items from a large pool.
From the test bank provided by the publisher.
Linked items to pages in the text, to facilitate study.
Encouraged multiple attempts
Recorded the highest grade on those attempts
This encourages students to keep taking the quizzes.

10. The effects of this strategy were obvious.
DFW rate dropped from 25% to 17%.
Students reported spending about twice as long preparing for classes.
There were 10-20% (depending on the chapter) more correct answers on common exam items.
Grades were about 10% higher.
Those students who were coming to class were better prepared.
More asked questions during class.

11. We have had some problems along the way
We had to change course management systems several times.
We do not have IT staff to do stuff for us, so we had to learn the systems and how to use them for ourselves
We had to create our quizzes for ourselves.
This was a time consuming process, especially if one wanted to link questions to page numbers
Most major publishers now deal with these issues for us.
Not all of us want to use the same text

12. Not everything we did worked well.
One instructor tried using discussion in a 100 student class
Required discussion was too much to grade
Followed by optional discussion – worked better in terms of instructor time, but few students participated.
One instructor replaced web-based quizzes with in-class quizzes. Effect was dramatic – students did not do as well on the in-class quizzes and exam grades dropped.
One instructor replaced half of the lectures with optional discussion sections – only about 25% of students took advantage of them.

13. Attendance dropped in some sections.
Some instructors continued to take all of their exam questions from the test bank.
Social pressure kept a few students coming to class – lectures were “entertaining.”
About 60% of students were not in class on any given day late in the semester.
In these sections there was no incentive for coming to class.
But it takes time to develop exam questions based on what was said in class.

14. Another problem: We had to rewrite our lectures.
We found that we had been enabling students to not read the book by reading the book to them
We knew that they had not read the assignment, so planned on reviewing the material.
They know that we would plan on their not reading so they did not bother.
Without having to read the book to them, we had to add stuff to our lectures.
One instructor cut seat time by 50%, others to a lesser degree

15. We then incorporated clickers into the introductory course.
A clicker is a device that transmits student responses to a receiver.
The receiver is connected to a computer.
Software records the responses
Which clicker sent the response.
What response was sent.
The aggregated responses can be displayed to the class.

16. Observations on the use of clickers
Radio frequency work much better than infra-red.
They can increase student attention span.
They break up the monotony of note taking.
Can increase the amount of active learning in a lecture environment.
They can provide feedback to students and the instructor.
Indicate who gets it and who does not.
Clicker questions can easily be included on exams.

17. Some concerns about clickers
How you use them is important.
Providing feedback as to why an answer is incorrect/correct takes time but is often worth it.
Daily use is important – it helps students to remember to bring them to class.
How you incentivize their use is important.
Rewarding “attendance” versus rewarding correct responses.
Random responding is a problem in large classes
Students may bring their friend’s clicker to class

18. Clickers filled the time we needed because of quizzing.
Forced faculty to rewrite their lectures to include clickers.
The time required to use clickers replaced most of the time from not having to cover the material we once did.
Some faculty found that they had less time to lecture than they now needed!
Some faculty spent hours getting their PowerPoint presentations perfect.

19. Clickers have been effective in Introductory Psychology
More students come to class when they receive credit for doing so.
Students report that they are better able to pay attention through the entire lecture.
More students answer clicker questions correctly on the exams than they did in class.
Exam grades went up about 10% overall, even on questions that were not clicker questions.

20. The combination of quizzes and clickers
DFW rates are down to about 12% (from 25%).
Student grades are better (more A’s, fewer D’s and F’s)
Student satisfaction with the course is much higher.
Faculty satisfaction with the course is much higher.
We are able to enlarge sections, freeing faculty to teach other courses.

21. If you want more details, please contact me directly.
John Broida
Psychology Department
University of Southern Maine
PO Box 9300
Portland, ME 04104