Evaluating Individual Contribution Toward Group Software Engineering Projects J. Huffman Hayes, T. Lethbridge, D. Port, USC Int’l Conf. on Software Engineering 2003 Portland, Oregon 5/8/03

Outline Group Projects – Characteristics – Benefits Related Work Grading individual effort – Criteria – Techniques – Bad things Best Practices Conclusions and Acknowledgments

Group Projects - Characteristics Staple of S/W eng. courses Teams of 3 or more Perform s/w development lifecycle One or more semesters May have real customer Develop artifacts

Group Projects - Benefits Learn to work in groups Learn from each other Learn to schedule Learn to rely on others Allows more complex systems

The Challenge Grading individual effort Handling “drop outs”

Related Work Speck [7] – group projects as part of “cooperative learning” Johnson et al [3] – essential factors for effective group learning – Positive interdependence – Effective interpersonal skills – Accountable

Related Work (continued) Schultz [6] found motivating slackers a concern Gates et al [2] – motivate students to contribute equally McKinney [5] suggests combination of: – Division of Labor report – Group grade – Assessment of dossier – Peer ratings

Grading Individual Effort - Criteria Fair Consistent Reflect achievement of educational objectives Provide good, understandable feedback Encourage student, avoid discouragement Easy on grader Control grading validity

Grading Individual Effort – Criteria (continued) No grade inflation/deflation Accurate and unbiased Discourage “risk managing” by students

Techniques/Grading Schemes 1) Group mark for all 2) Division of labor (DOL) report, then separate marks for parts 3) DOL, then grade relative contribution 4) In class pop quizzes on project 5) Cross validate w/results of individual work

Criteria to Grading Technique Map

Combining Schemes Fixed % of individual grade based on scheme 2) or 3), remainder on 1) S1*0.9 + S3*0.1 Group grade of S1 multiplied by a factor from S3 S1 * S3/100 Distribute group grade points – Group grade = 90%, so 90*3 points spread by 3 students NOTE: Spell out your intent in the syllabus

When Bad Things Happen Encourage students to work it out Act as intermediary Use Industry model (Chen [8])

Suggested Best Practices Allow peer evaluation, but monitor Use demos/quizzes to test project knowledge Have students maintain/post individual effort info Cross validate individual effort evaluation with group evaluation

Suggested Best Practices (continued) Use multiple methods for grading individual effort Evaluate grading scheme using criteria

Conclusions and Acknowledgments Determine your grading goals Establish a scheme Evaluate scheme Thanks to: – Hong-Mei Chen - U. of Hawaii – Tony Baxter, Paul Piwowarski, Jerzy Jaromczyk - U. of Kentucky

References [2] Gates, A.Q.; Delgado, N.; Mondragon, O., “A structured approach for managing a practical software engineering course”, 30th Frontiers in Education conf, IEEE, 2000, pp. T1C/21 -T1C/26. [3] Johnson, D. W., Johnson, R. T., and Smith, K. A. “Cooperative learning: Increasing college faculty instructional productivity”, ASHE-ERIC Higher Education Report 20, 4, Graduate School of Education and Human Development, The George Washington University, [5] McKinney, K. “Tips for Grading Group Work”, Illinois State University, web page as of Sept. 2002, [6] Schultz, T.W., “Students assessing teams”, proc. 29th Frontiers in Education conf, IEEE, 1999, pp. 13B2/1 -13B2/3. [7] Speck, B.W., “Pedagogical Support for Classroom Collaborative Writing Assignments”, ASHE-ERIC Higher Education Report, 28, 6, Jossey-Bass, a Wiley Company, 2002, pp [8] Chen, H-M., web page

Back-up slide – NEEDED? Or move in line? Evaluation of techniques – what has worked at Ottawa, USC, UK