1. Phil Davies School of Computing University of Glamorgan “Super U”
The Automatic Generation of ‘Marks for Marking’ within the Computerised Peer-Assessment of Essays
Phil Davies
School of Computing
University of Glamorgan “Super U”

2. Computerised Peer-Assessment
CAP System
Permits students to mark & comment the work of other students .
Also self-assess … used as a standard of expectation
Internet, not Web-based system (developed in Visual Basic / Access)

4. Having done the marking, what next?
Students should receive feedback
What feedback?
Marks
Comments
Which is most important?

6. AUTOMATICALLY EMAIL THE MARKER .. ANONYMOUS

7. What should the marker do? Reflect
Look at essay again
Take into account the essay owner’s comments
Further clarification (if it is needed, then is this a ‘black mark’ against the marker?)
Try to ‘appease’ the essay owner?
Modify mark based upon reflection?
Give more feedback

9. Must be rewarded for doing the ‘mark for marking’ process
How to judge?
Standard of expectation (self-assessment)
Marking consistency
Commenting, quality, measure against mark
Discussion Element
Need for additional comments – black mark?
Reaction to requests / further clarification

10. Does this student deserve a good mark?
ESSAY
Compensated Median Mark??
Student Mark
Difference W
65%
67%
+2% X
48%
54%
+6% Y
53%
57%
+4% Z
61%
59%
-2%
+2% on average

11. Standard of Expectation Self-Assess = 68% Peer-Assessed as = 58%
ESSAY
Compensated Median Mark
Student Mark
Difference W
65%
67%
+2% (8) X
48%
54%
+6% (4) Y
53%
57%
+4% (6) Z
61%
59%
-2% (12)
+2% (7.5)

13. How easy is it to do?
Statistically fairly easy to create a mark for marking based upon marks
Take into account high and low markers
Standard of expectation
Consistency … judge against final mark awarded for an essay (compensated median)
What about the comments?

14. Feedback Index
Produce an index that reflects the quality of commenting
Produce an average feedback index for an essay
Compare against marker in a similar manner to marks analysis
Where does this feedback index come from and is it valid?

15. The way to get the feedback index?
Develop an application??
C-Rater?
Spelling Mistakes
Similar Meanings?
That was cool
Really Choc
Really Good Essay
Manually

24. Differences from actual: F/Index Mark Score
DAI SMITH ESSAY /10
ESSAY /10
………… … … …
FINAL MARK AWARDED /10

25. Time Consuming? Can we formulate the marking process
Take away need for quantification process of analyzing comments
Is it still peer-assessment if the students are told what to say?

27. STUDENT FRED
REFERENCES: Positive ………
Negative …….
Personal Valuation
5, 3, 2, 1
3, 1, 2

28. Some points outstanding
What should students do if they identify plagiarism?
Is it ethical to get students to mark the work of their peers?
Is a computerised solution valid for all?
At what age / level can we trust the use of peer assessment?
How do we assess the time required to perform the marking task?

29. The Automatic Generation of ‘Marks for Marking’ within the Computerised Peer-Assessment of Essays
Student Acceptance:
Anonymity / Discussion / High & Low Markers
Tangible Reward
I’m more concerned with rewarding ‘higher order skills’
Getting there