1. The Effect of Formative Assessment in Brazilian University Physics Courses Omaha, August 1 st, 2011 Émerson Cruz and Gerd Kortemeyer Michigan State University

2. Émerson Cruz Study carried out by my graduate student Émerson Cruz Cannot be here today In Brazil collecting data

3. Formative Assessment Most post-secondary physics courses in Brazil offer no meaningful formative assessment ◦ no feedback from assigned homework ◦ traditional “transmission mode” lectures Implementing ◦ online homework with immediate feedback ◦ clickers Studying ◦ faculty and student acceptance ◦ effect on grade distribution

4. Collaboration Partners

5. Implementation Hurdles Our educational study quickly turned into a case study of implementation hurdles ◦ Public institutions:  grading structure based solely on exams  no possibility to give points for homework or clickers  exams very traditional and highly computational  no regular computer access for large percentage of students  no infrastructure to sell clickers and register them to individual students  had to distribute one classroom set to several sections at the beginning of lectures  no correlation data between clickers and grades  no reward structure for correctly answering questions ◦ Private institution:  Smaller statistics  Highly non-traditional student population (professionals)

6. No hard data yet Anecdotes Results from these two institutions Collaboration Partners

7. USP University of São Paulo ◦ one of the three public universities funded by the State of São Paulo. ◦ one of the largest institutions of higher education in Latin America ◦ approximately 75,000 enrolled students

8. Unianhanguera Unianhanguera ◦ one of the biggest private universities in Brazil ◦ more than 50 campuses spread out in seven Brazilian states ◦ largely non-traditional student population (professionals)

9. Usage USP ◦ Two sections of physics for engineers course  other sections traditional with different instructors  additional comparison: same instructor in previous years ◦ Online homework (LON-CAPA)  only 54 of the 157 accounts used  no grading incentive  76 of the 157 students said they used online homework  working in groups  access problems ◦ Clickers  given out at beginning of lectures  no data collected on correctness, participation, etc. Unianhanguera ◦ Two-month course ◦ Online homework  one section with and one without online homework (same instructor)  grading incentive: bonus points  all 55 students worked on online homework ◦ No clickers

10. Survey Results Gave a survey in both settings on usage habits and perceived helpfulness

11. First Action when Encountering New Problem

12. USA and Brazil: ◦ Male students tend to immediately try out a solution before reading up on the topic or seeking help  “Real men don’t read manuals or ask directions” Brazil: ◦ Students do not communicate with lecture personnel ◦ Brazil, private institution:  Students rarely read up on topic  Seek more help from peers First Action when Encountering New Problem

13. Additional Time-on-Task versus Perceived Helpfulness USA and Brazil: ◦ Online homework mostly perceived as helpful Brazil: ◦ Students report less time-on-task  Perception that online should be more time-efficient?  Sense of security due to more clearly defined expectations leading to less studying time?

14. Clickers Found broad student acceptance in survey at USP Instructor reports that lectures became more lively and interactive ◦ But: no hard data

15. Effect on Grades USA: ◦ particularly helpful for students who are about to fail the course Brazil: ◦ no significant difference ◦ USP: both with parallel sections (different instructors) and with sections taught previously (same instructor) ◦ Unianhanguera: with parallel section (same instructor)

16. Why no effect? Discrepancy: ◦ Students perceived online homework and clickers as helpful  Novelty? ◦ No significant difference in grades  Unable to reproduce USA results Why? ◦ Systemic change needed  value-system of a course is communicated by what is graded and on exams ◦ Many logistical hurdles in implementation  including lack of student internet access Only just beginning!

17. Acknowledgements Thanks to Prof. Hélio Dias at USP Thanks to i>Clicker for loaning us a classroom set of clickers korte@lite.msu.edu efcruz@msu.edu