1. May Faculty Institute (05/16/06) WeBWorK: The Ultimate Teaching Assistant? Olly GotelJackie Baldwin Rick KlineEileen Crupi Christelle ScharffTabitha Estrellado Andy Wildenberg * *

2. May Faculty Institute (05/16/06) Agenda Introductions and survey of online tools WeBWorK: –What it is, where it is used, what you can do with it, what we are doing with it and why Using WeBWorK as a CS student: –Range of questions it supports (with extensions) –Randomization and attempts Be the universal instructor/student: –Question formulation, answer entry and getting feedback (as both an instructor and student) Lessons to date Future work

3. May Faculty Institute (05/16/06) What Is WeBWorK? Web-based, automated problem generation, delivery and grading system Free, open-source project funded by NSF Initial development and applications in the fields of mathematics, physics Currently in use at more than 50 colleges and universities

4. May Faculty Institute (05/16/06) How Can It Help? Homework, Quizzes, Practice Randomization Instant feedback Ability to “try again”

5. May Faculty Institute (05/16/06) Who Else Is Doing This? WeBWorK: math, physics OWL (Thomson Course Technologies): chemistry (and perhaps other fields) CodeLab, CourseMarker, Codewitz, Gradience

6. May Faculty Institute (05/16/06) WeBWorK at Pace Extend WeBWorK for use in computer science curriculum Interface WeBWorK with other tools to facilitate grading of new problem types Enhance feedback mechanisms for both students and instructors Work funded by NSF grant

7. May Faculty Institute (05/16/06)

8. Using WeBWorK in CS121/IS223 Fall 2005 (Java): –For in-class tests (multiple choice / matching) Spring 2006 (Python and Java): –For in-class tests (multiple choice / matching) –For homework exercises TEACHING & LEARNING: LESSONS FROM EXPERIENCE

9. May Faculty Institute (05/16/06) Preparation More time upfront, less after (currently) Mitigating plagiarism: random ordering of answers in multiple choice questions, parameterization of code in matching questions; x out of y questions Importance of quality assurance Logistics: check clock synchronization, printing individual test papers

10. May Faculty Institute (05/16/06) Student’s Perspective Positive: –Instant feedback and self-assessment Issues and questions: –Consequence of no hardcopy: annotations, answer elimination? –Matching questions: text box size, multiple answers? –Context: % complete, visibility of progress? –Single attempt: answer commitment (shock to see results at the click of a button: where did I go wrong / so what?) –Multiple attempts: can lead to trial and error –Transfer of understanding between own programs and variants: demonstration of real understanding? –Motivation to use

11. May Faculty Institute (05/16/06) Professor’s Perspective Positive: –Class size (a helpful teaching assistant!) -- marking –Can see who is attempting homework –Variation in the questions –Quite straight forward to change questions to other programming languages –Reuse: next subjects in Cambodia and perhaps Finland! Negative: –Time to write and QA –Granularity of feedback for me and students –Going through solutions in class –Effective?

12. May Faculty Institute (05/16/06) Future Work Tailor questions dynamically to support specific student needs (to challenge or support) based on graph of topics Compare mechanism for student learning: references, hints, easier/harder questions Need more granular and visual feedback on performance for students and professors Possibility for class alerts: any common problems, trends? Measuring whether these systems actually work!

13. May Faculty Institute (05/16/06) Dissemination Poster Presentation at the Special Interest Group on Computer Science Education Conference, Houston (SIGCSE 2006) Poster Presentation at the Innovation and Technology in Computer Science Education Conference, University of Bologna, Italy (ITiCSE 2006)

14. May Faculty Institute (05/16/06) Your Questions ?

15. May Faculty Institute (05/16/06) Our Questions …