1. Effective Feedback, Rubrics, and Grading
Pamela Rist
Instructor in Epidemiology
Harvard T. H. Chan School of Public Health

2. My Teaching Experiences
Tend to teach in larger introductory level epidemiology or integrated epidemiology and biostatistics courses
Often have several TAs who are all working together to help grade the homework
Need to be consistent across graders and also make sure the TAs grade the same way I would (if I had the time to grade all the assignments myself)
Important to give the students high quality feedback since topics learned in courses I teach are the foundations for future classes

3. Giving effective feedback
Teaching and assessment are interconnected concepts
Important to remember that giving feedback is not just to “rank” students – can also be an important way to judge how effectively the information is being relayed and if the students understand the material
What would you like to see if you were the student?

4. General Tips Answer keys Writing comments on papers Be consistent
Turn-around times
Reviewing common mistakes
Giving feedback during seminars/labs

5. Grading Keys Useful for problems sets or short response questions
More detailed than an answer key
Used by the TAs to help ensure consistent grading
Highlights the “must haves” in an answer
Provides advice for how to grade common mistakes
Examples

6. Example Grading Key for problem on calculating and interpreting IRRs
b) Calculate the incidence rate among the exposed. (1 point)
Take off ½ point if they forget units
Give full credit if set-up is right but final number is wrong
Partial credit if some part is wrong (but base off the 2x2 table above – if that was incorrect, it’s OK to use the incorrect numbers here)
Incidence rate = 30 cases /63,359 person-years = cases/person-year
=4.73 cases/10,000 person-years
e) Interpret the incidence rate ratio in words. (1 point)
Take off ½ if no reference group is stated
Make sure they say “rate” not risk or odds (write in correct answer)
Heavy smokers have 1.76 times the rate of developing CHD compared to non-smokers.

7. Example of a Grading Key for a Short Answer Questions from a Study Critique
What was the outcome of interest? How was the outcome assessed? How complete was follow-up information? (3 points)
1 point for outcome
1 point for how assessed (self-report for non-fatal events; vital statistics, NDI or family/post office for fatal; confirmed by med record review --- take off ¼ if any of these three major pieces is missing)
1 point for completeness – take of ½ if they only talk about fatal or nonfatal
Do you believe that the findings observed in this study are valid? How did the authors address chance, bias, and confounding as potential alternative explanations for their results? (4 points)
Need to address chance, bias, and confounding – 1 point each
Should also have a conclusion about whether they think it’s valid – if they don’t take off ½ point
May get a range of answers on this one – make sure it’s reasonable and take off ½ points as needed if the logic does not work or if it’s under the wrong heading (putting a confounding scenario under chance for example)

8. Try on your own
The investigators indicated that they “adjusted” for a number of factors in their multivariate analyses. What factors did they adjust for? What does it mean to “adjust” for a factor? And what were they trying to accomplish by doing so? (2 points)
Things to think about:
What are the “must haves”? How much should each “must have” be worth?
What would be some common errors?

9. How would you grade these answers?
They adjusted for age, BMI, smoking, parental history of MI, alcohol use, history of hypertension, and physical activity. They “adjusted” for these factors to control for confounding by these factors.
The authors adjusted for lifestyle factors and medical history. They were trying to improve the generalizability of their results.
They adjusted for age, BMI, smoking, parental history of MI, alcohol use, history of hypertension, and physical activity. They were trying to control for selection bias and make the two groups more alike with respect to any baseline differences in these potential confounders.

10. Rubrics Another method to help ensure consistent grading
Find these helpful for grading group work, discussion board posts, papers, posters, etc.
Different formats
Often have a lists of dimensions or tasks that need to be covered and how much each contributes to the overall grade
Can also include guidance on how to assess each area (for example, 3 point scale or a 5 point scale)

11. Example of Components in a Poster Project Rubric
Background (1 point)
What is the current state of knowledge?
What is the specific gap in knowledge to be filled by this study?
Study Objective (1 point)
State your main objective
Methods (4 points)
State your study design
Describe your study population
Describe the definition of your exposure and outcome and how they were measured
Describe how the exposure and outcome were treated in the analysis. Were they categorized? Did you treat them as continuous variables?
Describe your main analysis plan. What statistical tests will you perform? If you use regression, state the type of regression used and the covariates included in the model.
Address how chance, bias, and confounding are incorporated into your analysis. For example, did you adjust for confounding? What confounders were included? Were the confounders linear or categorical variables?

12. Example of Components in a Rubric for a Final Paper
Criterion
Exceeds Expectations
Meets Expectations
Below Expectations
Demonstrated Knowledge
(50%)
Shows complete understanding of the exercise
Demonstrates full knowledge of the subject with explanations and elaboration
Points: 12.5
Shows substantial understanding of the exercise
At ease with subject content
Able to elaborate and explain to some degree
Points: 9.375
Response shows some understanding of the exercise
Uncomfortable with content
Only basic concepts are demonstrated and interpreted
Points: 6.25
Requirements
 (30%)
Goes beyond the requirements of the exercise
Points: 7.5
Meets the requirements of the exercise
Points: 5.625
Does not meet the requirements of the exercise
Points: 3.75
Report Format
(10%)
Format is consistent throughout, including heading styles and captions
Figures and tables are presented logically and reinforce the text
Points: 2.5
Format is generally consistent, including heading styles and captions
Figures and tables are neatly done and provide intended information
Points: 1.875
Mostly consistent format
Figures and tables are legible, but not convincing
Points: 1.25
 Adapted from: