How do we assign the final grade? Dheeraj Sanghi IIT Kanpur

Feb 2011Indo-US Workshop: Effective Teaching2 Agenda Why grades and not marks How many distinct grades Different ways of grading Conclusions

Feb 2011Indo-US Workshop: Effective Teaching3 Why grades, not marks From a student’s perspective Reduced stress Performance metrics are extremely important for students If slightly more work can result in better performance metric, the student is under pressure If a lot of hard work is needed to improve performance metric, then student is not likely to attempt it Assumption:  Student has an estimate of what value of performance metric she is going to get at a certain level of work Therefore:  Grading system works better when there is continuous evaluation

Feb 2011Indo-US Workshop: Effective Teaching4 Why grades, not marks From a teacher’s perspective Reduced stress It is extremely difficult to have the complete evaluation in a very objective and consistent fashion  Consistency here means that the result would not be very different, if a different set of questions were asked If the performance metrics to be assigned is continuous, any two students with a small difference, a different test could have resulted in a different ordering  For a broad performance metrics, several students with small differences will get the same metric, and consistency will be an issue only with borderline cases

Feb 2011Indo-US Workshop: Effective Teaching5 Are marks useful in any scenario? Marks (or performance metrics which take a large number of different values) are important in following situations: When the need to order students is very important For example, in admission tests like JEE  Having only grades in 12 th class exams would necessitate another exam for admission to programs where currently only marks are considered Would be nice to have standardized, objective tests to ensure reliability of evaluation When the evaluation is not for external consumption, but is being given as a feedback On an assignment, a student is not interested in just an overall grade, but where the marks are being deducted

Feb 2011Indo-US Workshop: Effective Teaching6 Should we only have Pass/Fail grades P/F grades will minimize stress both for students and teachers But lack of stress also means lack of incentive to learn Best learning takes place in an environment of moderate stress High stress causes exclusive focus on exams and not on learning Lack of stress encourages students to not take the course seriously Students won’t even participate in learning Teachers won’t get enough opportunity to instill the joy of learning that course.

Feb 2011Indo-US Workshop: Effective Teaching7 How many grades? Researchers in this aspect of assessment primarily study reliability of reporting How many students get the grades they “deserve” What is the average error in reporting performance Large number of grades: Would cause a higher mis-reporting, i.e., more students get a different grade than they deserve This affects credibility of the performance reporting system But the error in reporting performance would also be less for an average student Appears to be more fair If eventually reporting has to be done for a large number of examinations/courses The error will even out irrespective of the number of grades used Hence fairness is less important than credibility

Feb 2011Indo-US Workshop: Effective Teaching8 How many grades? If we further consider the impact of stress induced by the number of grades Less number of grades is to be preferred Except that the number should not be so small that it gives no incentive to learn Suggestion: Start with a small number of grades If students are not performing because there is too little stress, increase the number of grades If the students are not performing because there is too much stress, decrease the number of grades

Feb 2011Indo-US Workshop: Effective Teaching9 Different ways of assigning grades Mapping from marks to grades A pre-set mapping, for example: 90% and more get an “A” grade 75-90% get a “B” grade, etc. Gives no comparative information for a student across batches Gives no indication of broad rank within the class One can only compare two students of the same class with different grades

Feb 2011Indo-US Workshop: Effective Teaching10 Different ways of assigning grades Mapping from percentile to grades Again, a pre-set mapping, but not of specific marks, for example Top 25% get an “A” grade Next 25% get a “B” grade, etc. Gives some information about relative position of a student Someone with a “B” grade is behind at least 25% students in the batch If all courses employ a similar mapping then CGPA can give a good estimate of rank in the batch

Feb 2011Indo-US Workshop: Effective Teaching11 Different ways of assigning grades Find mean and standard deviation of the marks obtained by the entire class Use some mathematical functions to create a mapping from a range of marks to a specific grade, for example: Marks more than mean + SD get an “A” grade Marks between mean and (mean + SD) get a “B” grade

Feb 2011Indo-US Workshop: Effective Teaching12 Different ways of assigning grades To improve reliability (and credibility) of the grade Use one of the methods to first come up with rough grade boundaries Look in the neighborhood of that point, if there is a gap in the marks obtained by the candidates Larger the gap at the grade boundaries, better is the grade assignment from reliability perspective Chose the largest gap in the neighborhood and make this as the grade boundary Relative grading reduces the mis-reporting and further enhances reliability of the reported grade

Feb 2011Indo-US Workshop: Effective Teaching13 Who fails the course Assigning “F” grade is somewhat different from assigning other grades It should not be purely a relative grade At the beginning of the course, the instructor should articulate the minimum expectation from the course The evaluation throughout the course should be such that it allows to find out if that minimum learning has happened Grade “F” is to be assigned only if that minimum is not achieved

Feb 2011Indo-US Workshop: Effective Teaching14 Who fails the course However, there is a relative aspect of failure The minimum expectation should be established in such a way that not everyone is able to achieve that (assuming large class) If everyone meets that expectation, it shows that the expectations were low Good students would have been bored in the course The course could have been offered at a higher level Therefore, the instructor has been unfair to good students

Feb 2011Indo-US Workshop: Effective Teaching15 Conclusions Research in the area of grading tells what are the issues and how they relate to each other Generally there are no unique “answers” since the situation in different universities, cultures, classes, and disciplines is different But knowing the research, instructors, disciplines or university could come up with guidelines for improved assessment in their courses