Department of Early Childhood education Practices for Ensuring the Integrity of Assessment in Open and Distance Learning Salosh Govender Department of Early Childhood education Unisa govenp2@unisa.ac.za

OVERVIEW OF PRESENTATION Background the problem Threats to assessment integrity in an Open and Distance eLearning Assessment integrity Need for transformatiive assessment Rationale for transformative assessment Student graduateness Principles underpinning assessment integrity Transformative assessment strategies Key Stakeholders Validity, Reliability, Fairness

Threats to assessment integrity in an ODeL Context Absence of face-to-face (F2F) interaction,: Limited use of varied assessment methods and feedback to help students achieve the targeted learning objectives. Diverse student profile : Lack of authentic assessment activities to meet individual needs Limited opportunities to promote self regulated learning, online discussions, collaboration Inconsistencies in marking among our students between assignments and the examination Increased weighting of summative assessment in relation to formative Assessment focused on scoring rather than improving the quality of teaching and effective learning. It should be based on making expectations explicit and public, setting appropriate criteria and high standards for learning quality, systematically gathering, analyzing and interpreting evidence to determine how well performance matches expectations and standards, and using the resultant information to document, explain, and improve performance. Getting these variables to work in ODeL presents mammoth challenges. RESEARCH QUESTIONS How can ODeL lecturers validate and authenticate students’ written work? How can lecturers tell whether the students’ submitted work sufficiently reflects their knowledge and understanding? assessment at UNISA. What assessment practices contribute to assessment integrity within an ODeL What are the threats to the integrity of assessments in an ODeL context?

Assessment integrity WHAT IS ASSESSMENT? Process used to identify, gather and interpret information and evidence against the required competencies in a qualification in order to make a judgement about a students’ achievement. Can be formal, informal, non-formal Can be of learning already done, or towards learning to inform and shape teaching and learning still to be done ASSESSMENT INTEGRITY Honesty and transparency in every part of the assessment process. Assessment questions/ tasks must be based on work already covered Students must at all times be honest about what they offer to be assessed. Markers must understand what is offered by students for assessment and to ensure fairness in assessment. Moderators must moderate a fair sample of examples against a fair range of cases Core values of assessment integrity include : Constructive alignment Validity Reliability Accurate and consistent marking

Need for transformative assessment What is transformative assessment (TA) ? Refers to the continuous revision and review of assessment practices so that assessment practices are aligned to institutional goals, objectives, systems, programme outcomes and student profiles with the aim of improving student learning (Popham, 2017, Black & Wiliam, 2007) TA is driven by the extent to which academics engage in innovative assessment methods and practices – and the extent to which the stability of ICT / technology systems allows for innovation TA focuses on assessment methodologies, quality of the assessment instruments and the integrity of the assessment (Popham, 2017)

The rationale for transformative assessment within our ODeL environment Assessments are dominated by traditional methods of written assignments, tasks and tests. Assessment methods and types does not cater for diverse student profiles (inclusion, technology savy students, etc) New technologies create opportunities for innovative and alternate assessment tools. Improve validity, reliability authenticity of assessment Harness multiple stakeholder support in enacting assessment (eTutors, markers, teacher assistants, etc)

A key institutional goal: Student graduateness All programmes should aspire to develop students as global graduates capable of leadership and professional practice in a global community. Graduates should be capable of applying their discipline in local, national and international contexts Culturally aware and capable of respecting and acting in socially just/ responsible ways Capable of acting in environmentally responsible ways 

Principles underpinning assessment integrity Constructive alignment Validity Reliability Honesty

Transformative assessment strategies in ODeL Constructive alignment Authentic assessments Alternative assessments Key stakeholders Assessment for Learning Balancing formative and summative assessments

Theoretical underpinning: Constructive alignment model ( Biggs, 1999)   The curriculum is stated in the form of clear outcomes which state the level of understanding required rather than simply listing the topics to be covered. Develop an integrated assessment plan starting from the programme level exit outcomes Requires a co-ordinated process with fellow academics involved in teaching the module. The assessment tasks address the outcomes, so that you can assess to see if, and how well the students have learned what the outcomes state they should be learning. All components in such an aligned system thus address the same agenda and support each other. Ensures consistency that optimises the likelihood for students to engage in appropriate learning activities.

Formative Assessment “Formative assessment is a planned process in which assessment- elicited evidence of students' status is used by teachers to adjust their ongoing instructional procedures or by students to adjust their current learning tactics” (Popham, 2007) Formative assessment is not a test but a process—a planned process involving a number of different activities. One of those activities is the use of assessments, both formal and informal, to elicit evidence regarding students' status: the degree to which a particular student has mastered a particular skill or body of knowledge. Based on this evidence, teachers adjust their ongoing instructional activities or students adjust the procedures they're currently using to try to learn whatever they're trying to learn.

Assessment for Learning AfL Strategies Assessment for Learning techniques in an ODeL context Clarifying, sharing and understanding learning outcomes and success criteria Teaching plan of the study guide. Learning units Assessment tasks Use rubrics Providing feedback that moves students forward Key component for student interaction in ODeL context Supporting E tutors 2 and Reflective blogs 2 3 4 Markers feedback: constructive, specific and actionable Feedback on discussion forums Activating students as instructional resources for each other Plan authentic assessments, to engage students Encourage collaborative learning on discussion forums Activating students as owners of their own learning Self regulated learning Self assessment activities in study guide Rubrics to assist students to self assess Use the discussion forum platforms to establish students needs Rely on E tutors and markers feedback Case based activities Engineering effective class-room discussions, questions and tasks that elicit evidence of learning

Authentic Assessments Develops a sense of personal engagement Harnesses students' everyday experiences Brings a "live" feel to assessments Encourages students to focus/ reflect and evaluate their own learning Draws on and links to real world Links learning to practice Addresses real issues in the workplace Learning in and through communities Enhances students' perception of meaning and relevance of assessment Promotes deep approaches to learning Positions students as active learners Emulates complex ways of thinking Promotes hands-on experience of study

Transformative assessments linked to students needs OERS Curriculum responsiveness Student Reflection Context based Africanisation

A triadic relationship: Key stakeholders in ODeL Empower E tutors and markers Gather data from E tutors and markers Provide quality feedback to help students improve learning Academic eTutors Markers AfL Student – self assessment Students are empowered to take actions to improve their learning through self assessment Provide quality feedback

Validity and reliability of assessment practices Evidence showing that assessment practices fully cover the validity and reliability criteria Evidence of consistency in knowledge, skills, attitudes and values… Evidence of consistency across markers… Evidence that assessment is fit for purpose: Is fair Assessment measures intended abilities/ skills/ knowledge Assessment Criteria Use of rubrics All assessments are moderated( pre/ post moderation) Ensures alignment with NQF levels and cognitive levels Regular markers’ meeting Moderation of scripts Ensure markers have a good content knowledge: Markers complete exam papers Marking memo and rubrics Construct of valid estimate Construct validity/ content validity Assessments include questions from different cognitive levels Assessment include questions from different cognitive levels I moderate 10% of all scripts marked by marker Workshop academics in the department Member of the quality assurance committee QA through FTA Evidence