1. Umalusi Presentation to the
State of Readiness NSC 2011
Umalusi Presentation to the
Portfolio Committee 13 September 2011
Vijayen D Naidoo

2. Quality Assurance of the 2011 National Senior Certificate Examination

3. Regulatory Framework QAA NQF Act Section 27 (h)
The QC must develop and implement policy and criteria for assessment for the qualifications on its sub-framework.

4. Section17 of the GENFETQA Act
(5) The Council must, with the concurrence of the Director-General and
after consultation with the relevant assessment body or education
institution, approve the publication of the results of learners if the
Council is satisfied that the assessment body or education institution
has—
(i) conducted the assessment free from any irregularity that may
jeopardise the integrity of the assessment or its outcomes;
(ii) complied with the requirements prescribed by the Council for
conducting assessments;
(iii) applied the standards prescribed by the Council which a learner is
required to comply with in order to obtain a certificate; and
(iv) complied with every other condition determined by the Council.

5. Framework for QA of Learner Achievement
Based on established and existing practices in assessment for certification
Quality assures both prescribed components of External assessment (examinations) and Site based/ internal / continuous assessment
Use of systems, processes, and procedures to evaluate, inspect, monitor and report on examination systems, processes and procedures of public and private assessment bodies.

6. Framework for Quality Assurance of Assessment
Evaluation and /or accreditation of assessment bodies
Periodic inspection of assessment bodies
Ongoing monitoring of assessment bodies
Quality assurance of external examinations through:
Moderation of examination question papers
Monitoring the conduct of examinations
Moderation of marking
Standardization of assessment outcomes

7. Framework for Quality Assurance of Assessment
Quality assurance of internal assessment through:
Monitoring the implementation of internal assessment
Moderation of learner tasks and educator portfolios
Statistical moderation of internal assessment results.

8. Approval of Results - Policy
Compliance with minimum requirements
Shift from Moderation to Verification (Moderation for verification)
Emphasis on Assessment Body reports
The monitoring of the conduct of examinations and the moderation of marking as well as the monitoring of the implementation of internal assessment and the moderation of internal assessment is undertaken by Umalusi to verify the veracity of the assessment body reports in respect of these process.
Broadened definition of irregularities.

9. Ensuring Standards: Moderating question papers
PURPOSE
to ensure that the question papers are of the required standard
To ensure that the question papers are relatively:
- fair
- reliable
- representative of an adequate sample of the curriculum
- representative of relevant conceptual domains
- representative of relevant levels of cognitive challenge
External moderators (panels)

10. Scope of the quality assurance processes
Number of NSC 2011 question papers moderated
Target
Number of question papers moderated
Approved
DBE Nov 2011
129
92% (P2 languages still to be approved)
DBE Mar 2012
92%
IEB Nov 2011 76
100%
ERCO Nov 2011 40
88% (5 papers at third moderation)

11. Moderation of question papers
STRENGTHENING THE APPROVAL PROCESS
Additional layer of QA - final sign off
Participated in pre- testing of sample question papers
Post-exam analysis

12. Moderation of internal /continuous assessment
Internal assessment refers to any assessment conducted by the provider, the outcomes of which count towards the achievement of the qualification
panels of moderators / subject specialists
Moderation at all levels
Directives for internal assessment
Prescribed criteria
- Educator Portfolios
- Learner portfolios (body of evidence)

13. Moderation of internal assessment cont
Ascertain the appropriateness and standard of the assessment tasks being developed within schools
Ascertain the degree to which assessment bodies/provinces are attempting to ensure standardisation across
Determine the extent and quality of internal moderation and educator development
Determine the reliability and validity of the assessment outcomes

14. Moderation of SBA
Previous model - moderation of learner and educator portfolios at the end of the year.
Implementation of new model in 2010 –focus on tasks rather than on portfolios
Ongoing moderations across 9 PDE’s, IEB & ERCO
Improve feed back to educators
Improvement in SBA implementation

15. Planned Completed Still to happen Term 2/3
Number of subjects: 7 subjects sampled across PDE’s plus Life Orientation across all PDE’s
Subjects: Accounting, History, Geography, Life Orientation, Life Sciences, Mathematics, Physical Science, English FAL
(each assessment body was allocated 3 subjects + LO)
Term 2/3:
DBE Moderation was
conducted (4-6 July and
13-15 July). Umalusi
monitored the DBE
moderation.
Umalusi moderation
/verification was conducted
(11-21 July)
Mpumalanga & North West
re-scheduled
Term 4
3-19 October
ERCO
IEB
3-4 November
3-5 December

16. Moderation of SBA Non-adherence to sampling – EC
Provincial moderation still compliance/audit focused
Quality of tasks and assessment must be improved.
Training of Educators

17. Monitoring Administration of Exams
The focus is on:
Policy development and implementation
Systems, processes and procedures
Plan for assessment and moderation
Availability and training of staff

18. Monitoring of Examinations
Planned
Completed
Still to happen
State of readiness
Monitored the following DBE support visits to identified provinces.
Mpumalanga
KZN
Eastern Cape
Limpopo
North West
Free State
(22 September)
Follow up visits
to identified
PDE’s
Monitoring Conduct of Exams
Deployment of monitors and staff to examination and marking centres.
32 monitors (3- 4 per province)
9 Conveners and 23 Monitors:
4 days for writing the examinations
2 days for marking
4 staff members
3 visits each for writing phase
1 visit each for marking phase
Planning
17 October to 17 December

19. Verification of Marking
PURPOSE
Moderation of marking determines the standard and quality of marking and ensures that marking is conducted in accordance with agreed practices
Umalusi engages the following during the moderation of marking
1. Pre-marking/memorandum discussion: centralised memo discussions recommended - this will ensure consistency across marking centres
2. Off-site moderation of marking
3. On-site moderation of marking

20. Verification of Marking
Planned
Completed
Still to happen
Centralised verification of marking in 25 NSC selected subjects:
Post the paper being written

21. Verification of the Resulting Process
Planned
Completed
Still to happen
Subject structures
Subject structures verified.
Candidate registration
System verified during state of readiness visits.
Generation of mark Sheets
On-site during September 2011
Capturing of Marks
Standardisation data
Data sets received and input/comments submitted to DBE.
Awaiting test data sets from DBE.
Standardisation booklets
Booklets received and input/comments submitted to DBE.
Capturing of adjustments
Statistical moderation & resulting

22. Standardisation and Resulting
Planned
Date
Qualitative Input meeting
19th December 2011
Pre-standardisation meetings
21st & 22nd December 2011
Standardisation meeting
23rd December 2011
Verification of resulting processes
23-29 December 2011
Approval of results
30 December 2011

23. Monitoring the State of Readiness
New approach
One Assessment body (DBE) with 9 provincial arms vs. 9 separate assessment bodies.
Avoid duplication (parallel processes)and burden on provincial authorities
“Lift” the system – Umalusi to assume a verification role.
“Shadowing” the DBE team to establish the veracity of the DBE monitoring processes and to authenticate DBE reports.

24. The DBE Monitoring Process
Audit of PDE systems
PDE’s required to complete a self-evaluation instrument and develop improvement plan.
Monitoring visits (evaluation/support) to Head Office, District & Exam Centers- monitored against Audit report and improvement plan for Examinations and Assessment.
Submission of composite report to relevant authorities (Umalusi included)
Comprehensive approach that monitors exam systems (including SBA)

25. Umalusi findings-KZN Compliance / Strong points
Mature /consolidated exam system
Management and Operational plans
Mark capture system
Concerns
Staffing – vacant posts; contract workers
Security at nodal points ( outlying areas)
High number of marking centers
Irregularities

26. Umalusi findings-EC Compliance / Strong points
Mature /consolidated exam system
Management and Operational plans
Concerns
Staffing – vacant posts; contract workers
Security – storage facilities at Butterworth
Private Centers/independent schools
Large number of marking centers
Labour issues

27. Umalusi findings-Mpumalanga
Compliance / Strong points
Management and Operational plans
Key management posts filled
Exam processes in place
Concerns
Coordination and communication between HO and districts
Printing of Question papers
Private centers
Large number of marking centers
Security and
Appointment of markers

28. Umalusi findings-Limpopo
Compliance / Strong points
Management and Operational plans
50 posts filled
Exam processes in place
Concerns
Printing of Question papers
Large number of distribution points
Security
Appointment of markers
District monitoring
Script control

29. Umalusi findings-North West
Compliance / Strong points
Management and Operational plans
Assessment Management coordinating committee
Exam processes in place
Printing
Concerns
Use of schools as distribution points
Monitoring compromised – shortage of vehicles

30. Concerns: Other Provinces
Gauteng
Independent Schools
Western Cape
Highly centralised –minimal district involvement
Security of question papers – distribution and storage
Northern Cape
Security – storage of question papers; return of answer books

31. Umalusi Overall Evaluation
Systems and Processes are in place to ensure the administration of the 2011 NSC examinations.
Identified risks will have to be closely monitored. Includes follow up visits.
Umalusi remains confident that we will be able to ensure a credible 2011 NSC examination.

32. Thank you!