Assessor training 1

Welcome from AUA and NBS Graham Pitcher Why GBA and what is it? The basic matrix format Examples of how the matrices are used at NTU Key assessment regulations Ann Hartley The assessment titles for new cohorts The assessment matrix for KA1 Provision of good feedback Issues arising from moderation Referencing style Activity on the assessment matrix Q & A 2

Grade Based Assessment Part 1 - overview Grading Scheme for Postgraduate Taught Courses

What is Grade Based Assessment Grade based assessment describes the ‘quality’ of learning (Biggs, 2003) The term ‘distinction’ is used to capture the idea that a student with a distinction ‘thinks differently’ from a student with a pass. The descriptors of assessment criteria describe a level of achievement. 4

The NTU Grading Scheme for PG taught courses The generic level and grading descriptors are provided in the booklet, NTU ‘Grading Scheme for Postgraduate Taught Courses’. Generic level descriptors describe the level of achievement required by all NTU PG students. The general grading descriptors provide the standard of performance across a 14 point grading scale. 5

The Grading Scheme DISTINCTION (Excellent) Exceptional Distinction High Distinction Mid Distinction Distinction COMMENDATION (Very good) High Commendation Mid Commendation Commendation PASS (Good) High Pass Mid Pass Pass FAIL (Insufficient) Marginal Fail Mid Fail Low Fail ZEROZero00 ClassGrade Mark Range Numerical equivalent 6 Assessment A – Grade awarded Mid Pass (55) Assessment B – Grade awarded Mid Com (65) Weighting 40% - 60% Aggregate numerical equivalent = (40% x 55) + (60% x 65) = = 61 Grade for module = Commendation

So how does it work? Principle of constructive alignment (Biggs, J. (1999) Teaching for Quality Learning at University, Buckingham, SRHE/Open University Press) 7

Constructive alignment Assessment task Learning Outcome Incentive for higher achievement through graded assessment criteria Define threshold assessment criteria Learning and teaching strategy to enable learners to reach learning outcomes GBA What do we want students to be able to do at the end of the course, module, session etc.? What is the most appropriate way of assessing whether students can do what we want students to be able to do? What are the students expected to do in order to demonstrate that the learning outcomes has been achieved? What is the content and learning strategies that will enable students to acquire the necessary knowledge and skills. The grade defines the level of achievement. 8

Pass: award credit learning outcomes achieved Learning outcomes and GBA 100% 50% 0% Higher levels of achievement Learning outcomes partially or not achieved Student achievement 9

Grade Based Assessment Part 2 – Learning Outcomes and Assessment Criteria 10

Learning outcomes A learning outcome contains: A verb - what the learner is expected to be able to do; Words that indicate on what or with what the learner is acting; Words that indicate the nature of the performance required 11

Verbs for learning outcomes 12

Learning outcome At the end of the module students should be able to: Demonstrate a systematic understanding of customer profitability analysis and how it can be utilised to enhance the earnings per share of a commercial organisation. What the learner has to do: DEMONSTRATE What is the nature of the performance required: a systematic understanding of customer profitability analysis; how it can be utilised to enhance earnings per share In what context: enhancing earnings per share in a commercial organisation 13

Assessment task / criteria – To demonstrate a systematic understanding requires theoretical knowledge and practical knowledge. – To demonstrate how requires the use of some numerical analysis. – Appropriate task would be to apply CPA to a given situation and write a report to the Board of Directors. 14 Learning outcome: Demonstrate a systematic understanding of customer profitability analysis and how it can be utilised to enhance the earnings per share of a commercial organisation. 2 nd Learning outcome Communicate key issues and arguments in a written format to a professional standard

Assessment criteria Task - scenario provided with numerical element – Write a report to the Board of Directors critically evaluating the use of Customer Profitability Analysis as a means of enhancing the Earnings Per Share of XYZ plc. Criteria to be met by students – Report writing skills to professional standard – Use of appropriate theory/knowledge – Evaluation of CPA (benefits and drawbacks) – Relate to particular facts in scenario (Applied) – Appropriate numerical analysis 15 Learning outcome:  Demonstrate a systematic understanding of customer profitability analysis and how it can be utilised to enhance the earnings per share of a commercial organisation.  Communicate key issues and arguments in a written format to a professional standard Written communication Theoretical and practical elements, systematic understanding and context of commercial organisation. Applied in context and enhance EPS

Grade Based Assessment Part 3 – Completing the assessment matrix descriptors 16

Assessment matrix Header information Learning outcomes being assessed Assessment criteria Comments 17

Assessment matrix CriteriaLow Fail Mid Fail Marg. Fail PassCom’nDist’nExcept’l Dist’n Report writing skills to professional standard Evaluation of CPA (including theoretical knowledge) Relate to particularly facts in scenario Appropriate numerical analysis 18

Completing the grade descriptors The NTU general grading descriptors use the terminology of: Exceptional Distinction – Distinction – Commendation – Pass – Marginal Fail – Mid fail – Low fail – 19 Exceptional Excellent Very good Good Marginally insufficient Insufficient Highly insufficient

SOLO Taxonomy The NTU descriptors also utilise the SOLO taxonomy which was developed by Biggs and Collis (1982). 20

21

22

Completing the matrix - Pass standard Decide what constitutes a pass standard, i.e. what is adequate or satisfactory to achieve a pass standard. Once this is decided work up towards a Exceptional Distinction, then work down towards a Low Fail. It is an iterative process. You may find you need to split criteria into more specific ones, or even merge some together in order to distinguish the levels of achievement. Get a colleague to review it. Ask the students. 23

Assessment matrix CriteriaLow Fail Mid Fail Marg. Fail PassCom’nDist’nExcept’l Dist’n Report writing skills to professional standard Evaluation of CPA (including theoretical knowledge) Relate to particularly facts in scenario Appropriate numerical analysis 24

Appropriate numerical analysis Exceptional Dist’n Distinction Commendation Pass Marginal Fail Mid Fail Low Fail Numerical analysis is correct and used within report to illustrate points, but limited to basic calculations. Exceptional numerical analysis provided throughout the report to illustrate benefits and drawbacks and imaginative use of analysis (including sensitivity analysis) to illustrate the impact of Board decisions about strategic choices associated with customers on the EPS of XYZ plc. Excellent numerical analysis with sensitivity analysis provided to illustrate the impact of Board decisions about customers on the EPS of XYZ plc. Very good numerical analysis provided and used to highlight the impact of Board decisions relating to CPA on the EPS of the XYZ plc. Numerical analysis contains some errors that are significant to the results, but narrative reduces the impact of the errors. Numerical analysis contains significant errors and could result in misleading the Board of Directors in their decision making. Little or no numerical analysis offered. 25

Assessment matrix CriteriaLow Fail Mid Fail Marg. Fail PassCom’nDist’nExcept’l Dist’n Report writing skills to professional standard Evaluation of CPA (including theoretical knowledge) Relate to particularly facts in scenario Appropriate numerical analysis Evaluation of CPA (including theoretical knowledge and application to scenario) 26

Evaluation of CPA Exceptional Dist’n Distinction Commendation Pass Correct use of CPA model and good evidence of evaluation. Covers both benefits and drawbacks with explanation but limited to basic knowledge and obvious links to XYZ plc’s situation. An exceptional evaluation demonstrating significant insight and grasp of the scenario situation, highlighting the most significant benefits and drawbacks providing the Board with an objective appraisal of the situation and evaluation of the use of CPA to achieve their objective of enhancing the EPS. An excellent evaluation of the theoretical concept covering the benefits and drawbacks. Shows insight gained from the scenario information that provides the Board of XYZ plc with an excellent evaluation on the use of CPA to achieve the objective of enhancing the EPS. Very good explanation of theoretical concept and evidence of evaluation. A thorough coverage of benefits and drawbacks that utilised the information in the scenario effectively to provide the Board of XYX plc with a very good evaluation of the use of CPA to enhance the EPS. 27

Evaluation of CPA Pass Marginal Fail Mid Fail Low Fail Complete misunderstanding or no explanation of CPA. Benefits and drawbacks are incorrectly stated. No attempt to relate to the scenario information (basically ignored altogether). Correct use of CPA model and good evidence of evaluation. Covers both benefits and drawbacks with explanation but limited to basic knowledge and obvious links to XYZ plc’s situation. Some misunderstanding of CPA. Insufficient coverage of benefits and drawbacks or insufficient explanation. Very limited or no use of scenario information to link to XYZ plc’s situation. Misunderstanding of CPA. Very few benefits or drawbacks included, or some incorrectly stated. Very poor link to scenario information or no mention of XYZ plc’s situation. 28

Report writing skills to professional standard Exceptional Dist’n Distinction Commendation Pass Clear structure Good, appropriate writing style. Clear presentation of numerical analysis. Exceptional presentation of information and arguments that reflects a highly professional style of writing. Clear and objective presentation of arguments. Imaginative presentation of numerical analysis integrated into the report in a way that supports the arguments. Very effective use of supporting appendices. Excellent structure with information and discussion presented in logical, clearly defined sections that aided the understanding of issues relevant to XYZ plc. Written in a clear and objective, professional style. Excellent integration of numerical analysis with use of summaries supported by detail shown in appendices. Clear structure with very good use of headings and signposting of sections. Very good appropriate writing style. Clear presentation of summary numerical analysis in report with good use of appendices for detail. 29

Comments section At the end of the assessment matrix is a comments section. – Strengths – Areas for improvement (refer to guidance sheets) – Overall performance Stay on the lookout for updates as there are enhancements being made to the matrix that will be filtered through at a later date. 30

That completes the GBA presentation. Any questions? 31