Bieber et al., NJIT © Internship/Ph.D. Opportunity PLA: Participatory Learning Approach Michael Bieber Jia Shen, Dezhi Wu, Roxanne Hiltz Information Systems Department College of Computing Sciences New Jersey Institute of Technology September 2004

Bieber et al., NJIT © Motivation To increase learning of course content Learning through active engagement –involve students as active participants –with the full problem life-cycle –through peer evaluation Minimize overhead for instructors

Bieber et al., NJIT © PLA Process (e.g.,Participatory Exam) Each student creates 2 exam problems Instructor edits the problems if necessary Each student solves 2 problems Students evaluate (grade) the solutions to the problems they authored, writing detailed justifications Ph.D. students evaluate each problem a second time Instructor gives a final grade optional: Students can dispute their solution’s grade, by evaluating it themselves and writing detailed justifications Instructor resolves the dispute All entries posted on-line

Bieber et al., NJIT © Exam Process Control Assign ID Edit questions Assign who answers questions Assign level-2 graders Course Design Determine Final Grades Set up on-line environment Dispute final grade Level-1 and Level-2 graders grade solutions Make up problems Read - other problems - other solutions - grade justifications - disputes Solve problems Instructor Control ProcessStudent Learning Process Resolve Disputes Process Flow: Learning from doing the PLA activities additional learning from reading everything peers write

Bieber et al., NJIT © Exam Process Control Assign ID Edit problems Assign who solves problems Assign level-2 graders Course Design Determine Final Grades Set up on-line environment Dispute final grade Level-1 and Level-2 graders grade solutions Make up problems Confirmation ID, understand process Read - other problems - other solutions - grade justifications - disputes Solve problems Instructor Control ProcessStudent Learning Process Resolve Disputes

Bieber et al., NJIT © Enjoyability (2 semesters with CIS677) QuestionsSAANDSDMeanS.D.# I enjoyed the flexibility in organizing my resources 26.2%48.9%16.7%3.6%4.6% I was motivated to do my best work 23.5%42.9%28.2%3.4%2.1% I enjoyed the examination process 17.2%42.3%22.6%10.5%7.4% SA - strongly agree (5 points); A - agree (4); N - neutral (3); D - disagree (2); SD - strongly disagree (1); the mean is out of 5 points; S.D. - standard deviation Cronbach’s Alpha=0.68

Bieber et al., NJIT © Perceived Learning QuestionsSAANDSDMeanS.D.# I learned from making up questions 17.9%42.5%21.3%13.8%4.5% I learned from grading other students answers 17.7%48.1%19.4%9.3%5.5% I learned from reading other people’s answers 15.8%45.0%22.1%11.3%5.8% I demonstrated what I learned in class 13.6%50.2%22.6%10.9%2.7% My ability to integrate facts and develop generalizations improved 21.8%49.2%25.6%2.1%1.3% I learned to value other points of view 17.6%51.9%27.6%1.3%1.6% I mastered the course materials 7.4%51.6%31.4%6.9%2.7% Cronbach’s Alpha=0.88

Bieber et al., NJIT © Recommendation: Do Again! QuestionSAANDSDMeanS.D.# Would you recommend in the future that this exam process used? 20.7%40.1%24.5%8.9%5.8% Similar results for CIS365: undergraduate file structures course using short essay questions (Fall 2002)

Bieber et al., NJIT © PLA Environment Software Guide the process Form groups Assign problem solvers, evaluators, dispute arbitrators On-line templates to ensure full entries Guide people to post entries in correct place Incorporate group process tools Handle problems as much as possible –Remind people who are late –Reallocate who does what Based on a workflow management tool…

Bieber et al., NJIT © Extensions: Activity Support Can we build in tools to support each activity –How to construct a problem –How to research and construct a solution –How to evaluate a solution –How to resolve a disputed evaluation –Technical term: scaffolding (educational literature) –Initial approach: inquiry-based support (ed. literature)

Bieber et al., NJIT © Extensions: Full Collaboration Groups for: –Problems, solutions, evaluation, dispute arbitration Requires group process support –Process: work on each activity together or separately, internal review –Grading of individual group members –Process Tools: brainstorming, voting, etc.

Bieber et al., NJIT © Extensions: Scope Which activities? –so far: exams –what about: quizzes, homeworks, larger projects, in-class projects Which problem types? –so far: short and long essay questions –what about: multiple choice, short answer, computer programs, semester projects –Sub-problems: computer program design & implementation semester project outline & execution

Bieber et al., NJIT © Extensions: Scope, cont. Course Level –Graduate, undergraduate, secondary school (high school, junior high) Disciplines –IS/IT, business, science, engineering, humanities, medical, all of secondary school

Bieber et al., NJIT © Extensions: Scope, cont. Degree of Evaluation (assigning grades) –Currently: solutions –What about: quality of problems quality of evaluations/grades –All could be disputed Degree of Participation –students could evaluate each –students could arbitrate disputes

Bieber et al., NJIT © Internship/Ph.D. Opportunity Literature review and suggestions for all of the extensions For more information on PLA see