A Quantitative Investigation of SBA and Examination Scores in Adult Education in South Africa 13th SAAEA Conference Gaborone, Botswana Dr Nkoloyakhe Mpanza 19-22 May 2019

Presentation Outline Introduction and Background Purpose and Research questions Formulation of hypotheses Methodology Findings Conclusion

Introduction and Background Umalusi is responsible for the qualifications on level 1- 4 of the National Qualification Framework (NQF). The NQF Level 1 qualification for adults: provides second chance to adults and out of school youth who cannot be accommodated in the mainstream schooling sector.

Introduction and Background NQF Level ABET/Grade Level 5 - 10 HET 4 FET Grade 12 3 Grade 11 2 Grade 10 1 GET ABET Level 4 – Grade 9 ABET Level 3 – Grade 7 ABET Level 2 – Grade 5 ABET Level 1 – Grade 3 ABET Level 0 – Pre ABET

Introduction and Background Assessment in this qualification comprises Site-Based Assessment (SBA) and examination. The SBA and examination contribute 50% each towards the final mark. SBA scores are included in some of the high stake qualifications around the world (Williamson, 2016).

Introduction and Background Literature revealed inaccuracies regarding SBA in some countries Tailor (2005, in Williamson, 2016), statistical moderation generally adjusted SBA marks downward in the UK. van der Berg & Shepherd (2008) identified two types of assessment inaccuracies Assessment leniency, where SBA mark is far higher than an examination mark; and Low assessment reliability, where SBA marks and examination marks are weakly correlated.

Introduction and Background For Umalusi: SBA challenges also observed by Umalusi Statistical moderation used to validate credibility Difference above 10% between the means of SBA and that of examination indicate that the SBA is a poor signal of performance. Hence is considered misalignment Rejection of SBA scores

Purpose of the study & Research question The study was conducted to investigate whether the means of SBA and examination scores per learning areas that are assessed by the Department of Higher Education and Training (DHET) are comparable. Research Question Is the mean of SBA scores comparable to that of examination scores in each of the 5 learning areas assessed the DHET for the NQF Level 1 qualification for adults?

Formulated Hypothesis There is no significant difference between the mean of SBA scores and that of examination scores in the five learning areas assessed by the DHET.

Methodology This quantitative study was conducted using five purposefully sampled learning areas assessed by DHET. These learning areas are Small Medium and Micro Enterprises (SMME); Mathematical Literacy (MLMS); Mathematics and Mathematical Sciences (MMSC); Economic and Management Sciences (EMSC) and Natural Sciences (NATS). Source of data: November 2018 examinations and SBA submitted DHET to Umalusi.

Methodology Data was analysed using Statistical Analysis System (SAS). Two-sample t-tests were conducted per learning area to test the formulated hypothesis. The selected level of significance (alpha) is p= 0.05

Findings Learning Area Mean-SBA Mean-Exam Difference Between Means Degree of Freedom (df) P-Value SMME 59.175 33.357 25.189 7647 NATS 61.926 31.733 30.193 4415 MLMS 60.756 35.476 25.280 38014 MMSC 62.287 38.455 23.832 5359 EMS 62.223 26.166 36.057 10353

Findings Hypothesis “There is no significant difference between the mean of SBA and that of examination scores in the five learning areas assessed by the DHET.” Rejection rule: p-value < alpha then we reject. The p-value is less than alpha (0.05) in all 5 learning areas, therefore, hypothesis was rejected. There is a significant difference between the SBA and examination scores in all 5 learning areas

Conclusion The study concludes that: student performance in SBA and examination of selected learning areas is not comparable. This motivates for a continuation of statistical moderation process.

Thank You