1. 13th SAAEA Conference, Botswana 19-22, May 2019 Mapaleng Lekgeu
Towards a critical re-evaluation of school-based moderation processes in the era of accountability
13th SAAEA Conference, Botswana
19-22, May 2019
Mapaleng Lekgeu

2. Presentation outline Contextualisation of the study
Research purpose and question
Literature Review
Theoretical framing
Methodology
Discussion of findings
Conclusions

3. Contextualisation of the study
School-based Assessment (SBA) was introduced in 2001 in South Africa (SA) by minister of Education – Kader Asmal, after having been piloted in three provinces (WC,NC & GP), in
Objectives:
(1) to fragment the high-stakes, once-off examination system;
(2) to use assessment for formative purposes and to provide feedback timeously;
(3) to modify instructions; to diagnose and remediate areas of learners’ weaknesses.
Named Continuous Assessment (CASS) under Curriculum (OBE, 1998); but became SBA under National Curriculum Statement (NCS) in 2003 and Curriculum and Assessment Policy Statement (CAPS) in 2012.

4. Contextualisation Continues……
Moderation of SBA tasks became cumbersome:
site-based, district and provincial, quarterly;
done according to exam norms to meet standards;
thus highly summative, defeating the ends
Amid a plethora of policy directives, the problems of implementation of SBA moderation continued unabated.
Some of the problems are:
disregard of policy imperatives; with no uniform interpretation and application of policy directives
variations in the way schools implement moderation;
Lack of due diligence in moderation
quality of tasks affected;
marks inflated, performance obscured as a result, Poliah (2010)
Moderation criteria: CAPS-aligned content, cognitive levels, variety of questioning techniques, marking guideline and weightings

5. Contextualisation Continues……..
As reported by Umalusi evaluators,
poorly constructed tasks,
unreliable memoranda and
inconsistent mark allocations; could still be found in some of the schools
This study therefore aims to
re-evaluate SBA moderation processes and
to suggest ways of improving moderation and to hold moderators liable
The re-evaluation of SBA moderation relates to independent schooling sector in Gauteng province, South Africa
The definition of moderation, in its broadest terms, also receives the attention of the study.

6. Research Purpose and Question
to re-evaluate the SBA moderation process in such a way as to make moderators accountable and to suggest new ways of doing things .
Research Question:
How can SBA moderation processes be used for moderator accountability?

7. Literature Review
There are differences in the definition of moderation across the literature:
comparability, reliability and validity of teacher judgements (Grant, 2013)
communicative and collaborative process of judging learner portfolios (Van der Schaaf, Baartman & Prins, 2012)
quality control process involving monitoring, approval and judgements ensuring consistency in interpretation and application of standards (Maxwell, 2006)
process which ensures that assessment of outcomes are fair, valid and reliable (Umalusi, 2018)
This study adopts the definition as put forward by Maxwell (2006), i.e. that of moderation
as quality control involving monitoring of teacher judgements;
thereby ensuring conformity in interpretation and application of standards

8. Literature Review continues……
But the study further views moderation
as not only quality control but also quality assurance process;
a process in which teachers collaborate and corroborate in community of practice
In fact the study argues that moderation should be a five-stage process:
pre-assessment moderation
process moderation
verification of marking
verification of marksheets
feedback to teachers (developmental) and learners (diagnostic)

9. Methodology
Quantitative study involving content analysis of reports archived at Umalusi
The reports included data collected from forty (40) independent schools in Gauteng Province visited between 2016 and 2018
The schools were sampled purposively.
Schools were then grouped according to regions, namely Ekurhuleni, JHB and Tshwane for analysis purposes

10. Analysis of the five-stage moderation
Pre-assessment moderation:
the first stage of moderation which looks at CAPS content; cognitive levels; questions variety; mark allocation; intended skills and question analysis
Process moderation:
this is a stage where portfolios are moderated; irregularities managed; and actual writing of assessment tasks monitored.
Verification of marking:
this includes script sampling procedures (10%); checking of compliance to marking guideline; and learner moderation
Verification of marksheets:
moderators verify compliance to correct design; correct recording and weighting of marks; and mark analysis
Feedback to teachers and learners:
developmental feedback to teachers; diagnostic and enrichment feedback to learners

11. Findings

12. Findings continues…

13. Discussion of the findings
Compliance to the five-stage moderation process appears to be erratic in all schools across the identified regions
Pre-assessment moderation is popular in Tshwane region with 100% of schools complying
As much as 83% and 75% of schools in Johannesburg (JHB) and Ekurhuleni regions respectively comply with pre-assessment moderation
Less than 70% of schools across the regions comply with the criterion of moderation feedback to teachers and subsequently to learners.
The situation of compliance to moderation feedback is not good in Ekurhuleni where 58% of schools are non-compliant.
In fact, Ekurhuleni comes third in compliance to all criteria across the regions

14. Discussion continues…….
It appears that between 80 and 100% of schools in all regions comply with CAPS-aligned content, questions variety and marking guideline as criteria for pre-moderation
Less than 40% of schools conduct question analysis, thereby determining level of difficulty of questions before they are written.
A satisfactorily number of schools (over 65%), comply with the criterion of cognitive levels
Although over 50% of schools in JHB and Tshwane regions comply with the criterion of weighting of marks, merely 25% of them do so in Ekurhuleni.

15. Conclusions In conclusion, the following points are worth noting:
a broad-based countrywide study in the implementation of the five-stage moderation model is imperative
the five-stage model proved to be implementable in this study, thus productive and rewarding.
the actualisation of the suggested model of moderation varies between schools in all the identified regions
insufficient literature regarding moderation liability by moderators
schools, rather than moderators, should carry the responsibility of accounting for moderation
management of moderation processes should involve quality assurance, quality control and quality review

16. Recommendations
the district offices should establish an accountability office made up of district officials to adjudicate on laissez-faire moderation; proposing and reporting on plan of action
moderators should be mobilised into networking systems with colleagues in cluster of schools as community of practice – the idea of social moderation
create platforms where school-based moderators would interrogate moderation feedback from external moderators
introduce consequence management for underperforming moderators: assistance first and then be subjected to corrective action and ultimately sanctions
build teachers’ assessment literacy and capacity
incentivise good practices

17. Thank You