The Effect of Formative Assessment in Brazilian University Physics Courses Omaha, August 1 st, 2011 Émerson Cruz and Gerd Kortemeyer Michigan State University

Émerson Cruz Study carried out by my graduate student Émerson Cruz Cannot be here today In Brazil collecting data

Formative Assessment Most post-secondary physics courses in Brazil offer no meaningful formative assessment ◦ no feedback from assigned homework ◦ traditional “transmission mode” lectures Implementing ◦ online homework with immediate feedback ◦ clickers Studying ◦ faculty and student acceptance ◦ effect on grade distribution

Collaboration Partners

Implementation Hurdles Our educational study quickly turned into a case study of implementation hurdles ◦ Public institutions:  grading structure based solely on exams  no possibility to give points for homework or clickers  exams very traditional and highly computational  no regular computer access for large percentage of students  no infrastructure to sell clickers and register them to individual students  had to distribute one classroom set to several sections at the beginning of lectures  no correlation data between clickers and grades  no reward structure for correctly answering questions ◦ Private institution:  Smaller statistics  Highly non-traditional student population (professionals)

No hard data yet Anecdotes Results from these two institutions Collaboration Partners

USP University of São Paulo ◦ one of the three public universities funded by the State of São Paulo. ◦ one of the largest institutions of higher education in Latin America ◦ approximately 75,000 enrolled students

Unianhanguera Unianhanguera ◦ one of the biggest private universities in Brazil ◦ more than 50 campuses spread out in seven Brazilian states ◦ largely non-traditional student population (professionals)

Usage USP ◦ Two sections of physics for engineers course  other sections traditional with different instructors  additional comparison: same instructor in previous years ◦ Online homework (LON-CAPA)  only 54 of the 157 accounts used  no grading incentive  76 of the 157 students said they used online homework  working in groups  access problems ◦ Clickers  given out at beginning of lectures  no data collected on correctness, participation, etc. Unianhanguera ◦ Two-month course ◦ Online homework  one section with and one without online homework (same instructor)  grading incentive: bonus points  all 55 students worked on online homework ◦ No clickers

Survey Results Gave a survey in both settings on usage habits and perceived helpfulness

First Action when Encountering New Problem

USA and Brazil: ◦ Male students tend to immediately try out a solution before reading up on the topic or seeking help  “Real men don’t read manuals or ask directions” Brazil: ◦ Students do not communicate with lecture personnel ◦ Brazil, private institution:  Students rarely read up on topic  Seek more help from peers First Action when Encountering New Problem

Additional Time-on-Task versus Perceived Helpfulness USA and Brazil: ◦ Online homework mostly perceived as helpful Brazil: ◦ Students report less time-on-task  Perception that online should be more time-efficient?  Sense of security due to more clearly defined expectations leading to less studying time?

Clickers Found broad student acceptance in survey at USP Instructor reports that lectures became more lively and interactive ◦ But: no hard data

Effect on Grades USA: ◦ particularly helpful for students who are about to fail the course Brazil: ◦ no significant difference ◦ USP: both with parallel sections (different instructors) and with sections taught previously (same instructor) ◦ Unianhanguera: with parallel section (same instructor)

Why no effect? Discrepancy: ◦ Students perceived online homework and clickers as helpful  Novelty? ◦ No significant difference in grades  Unable to reproduce USA results Why? ◦ Systemic change needed  value-system of a course is communicated by what is graded and on exams ◦ Many logistical hurdles in implementation  including lack of student internet access Only just beginning!

Acknowledgements Thanks to Prof. Hélio Dias at USP Thanks to i>Clicker for loaning us a classroom set of clickers