The relation between assessment and learning Prof. dr. M. Segers. University Leiden (NL)

Wibran and the music class

‘Assessment steers learning’

Negative effects of traditional tests Theorized benefits of new modes of assessment Traditional tests: referred to as measuring low levels of comprehension; negative effects on various motivational aspects (Harlen & Crick, 2001 ) and stimulating the use of surface learning approach Traditional tests: referred to as measuring low levels of comprehension; negative effects on various motivational aspects (Harlen & Crick, 2001 ) and stimulating the use of surface learning approach new modes of assessment: measuring knowledge construction & application, problem- solving, reflection & critical analysis, going beyond mere facts, relating,…. Supporting the use of a deep approach to learning new modes of assessment: measuring knowledge construction & application, problem- solving, reflection & critical analysis, going beyond mere facts, relating,…. Supporting the use of a deep approach to learning

With the implementation of new modes of assessment, students are stimulated to focus on these goals. It is expected that, in comparison with traditional tests, this will lead to students’ adoption of a deep approach to learning and considerable learning gains Pre-assessment effect

Although the statement ’assessment steers learning’ has stimulated many schools to change their assessment practices, the empirical evidence of the effect of these changes on student learning is scarce.

3 empirical studies Segers, Nijhuis & Gijselaers: effect of redesign of learning and assessment environment on students’ learning strategies (International Business) Segers, Nijhuis & Gijselaers: effect of redesign of learning and assessment environment on students’ learning strategies (International Business) Segers, Gijbels & Thurlings: The effect of perceptions of the assessment practice on student learning approaches. The case of portfolio assessment (Applied Science) Segers, Gijbels & Thurlings: The effect of perceptions of the assessment practice on student learning approaches. The case of portfolio assessment (Applied Science) Segers, Martens, Nijhuis, Cats & Verwijmen: the role of assessment perceptions and prior experiences with assessment on student learning approaches (Teacher Training program) Segers, Martens, Nijhuis, Cats & Verwijmen: the role of assessment perceptions and prior experiences with assessment on student learning approaches (Teacher Training program)

Research model Perceptions Of the assessment demands Of the assessment practice Prior experiences Intentional learning approaches Personal beliefs on assessment Learning approaches

Segers et al: effect of redesign of learning and assessment environment on perceptions of the assessment demands and learning strategies Redesign from assignment-based format (ABL) to a PBL format: (1) from clear-cut study assignments to authentic business problems, (2) from passive participation to active discussion; (3) from one handbook to multiple literature resources (4) assessment: from knowledge reproduction (ABL) to use of knowledge as a tool to solve novel problems (PBL)

Case Mexx/Benneton (This case of approx. 35 pages is new the students) The case study presents the history and recent developments in the fashion company Mexx. Main trends within the European clothing industry are described. Mexx Fashion and Benneton are pictured by their organizational structure, the product profile and marketplace, the business system, the corporate culture and some actual facts and figures. Question1 true/?/false Mexx’s corporate culture and philosophy is consistent with the systems viewpoint on management. (False, it is consistent with behavioural viewpoint) Question 2 Benneton’s and Mexx’s corporate strategies are quite different. More specifically, there are two main differences. a.Identify these two main differences in corporate strategies. Illustrate your answer with examples mentioned in the case. b.What are the advantages of Benneton’s corporate strategy compared to Mexx’s approach?

Research question: Effect of redesign on students’ learning strategies? Relation with intentional learning strategies and perceptions of assessment demands?

Actual Learning strategies: R-SPQ-2F (Biggs) Actual Learning strategies: R-SPQ-2F (Biggs) Surface strategy: I learned some things by rote, going over them until I knew them by heart Deep strategy: I related material, as I was reading it, to what I already knew on that topic Deep strategy: I related material, as I was reading it, to what I already knew on that topic Intentional Learning strategies: R-SPQ-2F (Biggs) Intentional Learning strategies: R-SPQ-2F (Biggs) Surface strategy: I intend to learn things by rote, going over them until I know them by heart Deep strategy: I intend to relate material, as I am reading it, to what I already know on that topic Deep strategy: I intend to relate material, as I am reading it, to what I already know on that topic Perceptions of Assessment Demands (Scouller) Perceptions of Assessment Demands (Scouller) Assessment of surface learning skills: I expect the test to assess my ability to reproduce key terms and definitions Assessment of surface learning skills: I expect the test to assess my ability to reproduce key terms and definitions Assessment of deep learning skills I expect the test to assess my ability to integrate from a variety of sources Assessment of deep learning skills I expect the test to assess my ability to integrate from a variety of sources

ABLPBLP Actual deep learning strategies (3.28)22.22 (3.48)0.000 Actual surface learning strategies (3.11)17.98 (2.93)0.000 RESULTS Result 1: less deep learning in OAT-condition

ABL conditionActual deep LS Deep perception assessment demands Intention to employ deep learning strategies.433**.290** PBL condition Intention to employ deep learning strategies.428**.334** Result 2: Influence of prior experiences

Segers et al: the relation between learning approaches, perceptions of the assessment demands and personal beliefs on assessment demands Context: 4 year teacher training program; year 1& 2 knowledge reproduction tests; year 3 OverAll Tests; year 4 portfolio assessment Context: 4 year teacher training program; year 1& 2 knowledge reproduction tests; year 3 OverAll Tests; year 4 portfolio assessment Research question: what is the relation between student learning approaches and their perceptions of assessment demands and personal beliefs on assessment demands? Research question: what is the relation between student learning approaches and their perceptions of assessment demands and personal beliefs on assessment demands?

Personal beliefs of assessment demands Belief of surface assessment: In general I believe a test assesses my ability to reproduce key terms and definitions Belief of deep assessment: In general I believe a test assesses my ability to integrate from a variety of sources

Surface approach Deep approach Surface percep. Deep percep Beliefs surf. Beliefs deep Surface approach -,510**-,126-,321*-,083-,159 Deep approach,188,215,309*-,003 RESULTS Result 1: Some association between approaches and perceptions, beliefs

PERCEPTIONS SURFACE PERCEPTIONS DEEP BELIEFS SURFACE,544(**)-,004 BELIEFS DEEP -,051,604(**) Result 2: role prior experiences A strong positive association between perceptions of assessment demands and personal beliefs

Segers et al: The effect of perceptions of the assessment practice on student learning approaches. The case of portfolio assessment Context: 4 year program Applied Science; competency-based; portfolio assessment Context: 4 year program Applied Science; competency-based; portfolio assessment Digital portfolio: PAP, PDP, study dossier (work done and work in progress) and private dossier ; basis for the assessment dossier for end-of-year assessment Digital portfolio: PAP, PDP, study dossier (work done and work in progress) and private dossier ; basis for the assessment dossier for end-of-year assessment Research question: is there a relation between students’ perceptions of the portfolio assessment and their learning approaches? Research question: is there a relation between students’ perceptions of the portfolio assessment and their learning approaches?

ScaleItem examples 1. Amount and distribution of student effort I only study things that are going to be covered in the assignments; On this course it is possible to do quite well without studying too much 2. The portfolio assessment and learning Making the portfolio brought things together for me; The instructions are very clear about what you are expected to do for the portfolio assessment; The portfolio assessment is not very challenging 3. Quantity and timing of feedback On this course I get plenty of feedback on how I am doing; The feedback comes back very quickly 4. Quality of feedbackThe feedback helps me to do things better; I can seldom see from the feedback what I need to do to improve 5. What you do with the feedback I use the feedback to go back over what I have done in the assignment; I tend to only read the marks Perceptions of portfolio assessment (Gibbs Assessment Experiences Questionnaire, 2994)

Amount and distribution of effort -,118-,220*-,128-, Portfolio Assessment and learning,354**,402**,383**0.320**-0.267** 3. Quantity and timing of feedback,337**,189* * 4. Quality of feedback,317** ** 5. Student’s use of feedback 0.371**-0.395** 6. Deep learning approach ** 7. Surface learning approach RESULTS

Results: Students’ perceptions of the assessment practice are related to their approach to learning, more specifically, how they perceive the feedback and how hey perceive the relation between assessment and learning

Overall, what do these studies indicate? Simply changing the mode of assessment does not always result in changes in learning approaches Personal history of students plays a role: Personal history of students plays a role: (1) intentional learning strategies (2) personal beliefs on assessment Role of students’ perception of assessment practices Role of students’ perception of assessment practices

Implications for future research A challenge for future research is to gain insight into the choices a student makes when trying to deal with the various individual and contextual elements and the demands of the learning and assessment environment: the interplay between prior experiences, prior experiences, perceptions and appraisal of assessment perceptions and appraisal of assessment learning approaches learning approaches

Implications for educational practice Give student time and opportunities to practice and to have positive experiences Give student time and opportunities to practice and to have positive experiences Talk to students and observe them in order to gain insight in how student deal with the assessment practice in order to fine-tune learning and assessment practices Talk to students and observe them in order to gain insight in how student deal with the assessment practice in order to fine-tune learning and assessment practices Organize enough and on-time feedback and give students opportunities to work with the suggestions they have received Organize enough and on-time feedback and give students opportunities to work with the suggestions they have received Make the assessment a learning activity by giving them challenging assessment tasks and making the assessment process transparent Make the assessment a learning activity by giving them challenging assessment tasks and making the assessment process transparent

Or… Overcoming assessment challenges!