LANGUAGES The links between the: Standards Texts Questions and Schedules

The Curriculum MoE The Curriculum MoE Achievement standards MoE Achievement standards MoE The Grade score NZQA The Grade score NZQA The hierarchy Assessment Task NZQA Assessment Task NZQA A happy student A happy student 2

The listening standards Level 1 (1.1) Demonstrate understanding of a variety of spoken LANGUAGE texts on areas of most immediate relevance Level 2 (2.2) Demonstrate understanding of a variety of spoken LANGUAGE texts on familiar matters Level 3 (3.1) Demonstrate understanding of a variety of extended spoken LANGUAGE texts At all three levels: Merit requires ‘clear understanding’ Excellence requires ‘thorough understanding’. 3

The reading standards Level 1 (1.4) Demonstrate understanding of a variety of LANGUAGE texts on areas of most immediate relevance Level (2.4) Demonstrate understanding of a variety of written and/or visual LANGUAGE texts on familiar matters Level 3 (3.4) Demonstrate understanding of a variety of extended written and/or visual LANGUAGE texts At all three levels: Merit requires ‘clear understanding’ Excellence requires ‘thorough understanding’. 4

The standards, a summary The colours indicate Grade: AchievementMerit Excellence NCEA level:Level 1 Level 2 Level 3 The listening standard Demonstrate clear / thorough understanding of a variety of extended spoken Tongan texts on areas of most immediate relevance / on familiar matters The reading standard Demonstrate clear / thorough understanding of a variety of extended written and/or visual Tongan texts on areas of most immediate relevance / on familiar matters 5

The definitions that matter For Merit Clear understanding means the relevant information ideas and/or opinions from the texts are selected and unambiguously communicated. For Excellence Thorough understanding means relevant information, ideas and/or opinions from the texts are expanded on with supporting detail to show understanding of the implied meanings or conclusions within the text. 6

The text The text is the most important thing, it must contain: straight forward information (Achievement) material that can used to justify opinions / draw conclusions / infer / link / analyse (Merit and Excellence) Texts and questions can link in several ways: one text, one question one text, more than one question several linked texts, one question If the text is good, the questions ‘fall out’ of it easily.

The question The question should elicit sufficient information for the marker to be able to validly judge the level of understanding the student is demonstrating As you are writing / critiquing, bear in mind two students: the ‘just achieved’, literal, conscientious one the excellence one who thinks outside the box If your question can be successfully attempted by both those candidates, it will discriminate successfully at all levels. Scaffold if possible; that is step the candidates through by starting with a straightforward ‘A’ part, then following on with an M or M/E part, Avoid very broad ‘drag net’ questions; candidates at all levels tend to respond to them by writing everything. Be brief and clear, don’t ask more just because you can; ask for less, target more tightly, specify clearly.

The assessment schedule The schedule is formatted the same way for both standards, all languages, at all three levels. It is intended to enable holistic judgement for each question, and avoid focussing on counting up fragmented pieces of evidence / lexical items. It is a guideline to how to make judgements, not a black and white list of ‘right’ and ‘wrong’ answers. Judgements between grades (that is N and A, A and M, M and E) must be qualitative Judgements within grades may be qualitative or quantitative Although final cut-scores may vary from exam to exam (depending on relative difficulty) adherence to the intent of the standard supersedes all other considerations. That is, final judgements must be made against the standard. 9

Grade scores There are nine possible grade scores: NØ, N1, N2, A3, A4, M5, M6, E7, E8 Each question is marked as a ‘stand-alone’ The grade score awarded is based on: the criteria from the standard the entire response to the question (that is holistically) what the candidate does know and can do Step ups are: qualitative between grades quantitative and/or qualitative within grades The scores are totalled once all the questions have been marked 10

Grade scores in languages What to do and not do when deciding grade scores Concentrate on determining how well the candidate understood the text overall. Avoid focussing on individual lexical and grammatical items. Don’t count up bits of evidence; look at a response overall, starting with the E evidence. 11

Top-down marking All standards are marked using top-down marking which conforms to the fundamental principle on which standards based assessment sits. Top down marking is fairer and more valid for the candidates because: It recognises all evidence (including N1 or N2) and aggregates it across the paper. The absence of lower level evidence does not ‘knock’ a candidate out of a higher level of achievement.

The steps in top down marking Look for Excellence evidence; if there is sufficient to indicate the candidate is performing at E level: –award E –decide if it is ‘a good E’ (E8) or ‘just an E’ (E7) –write the appropriate grade score in the box provided –move on to the next question. If Excellence evidence is not there, look for Merit evidence If Merit evidence is not there, look for Achievement evidence If Achievement evidence is not there, decide whether to award NØ, N1 or N2 Total the scores and write the total on the front of the answer booklet.

Quick analysis of a test On one page, draw up a tally table like the one below. Go through the class results, one student and a time. Enter (as a tally) each student’s results for the four questions. You can now see which questions : are discriminating well seem to be too ‘hard’ or too ‘easy’ have a very strange spread. Score Question One llllllllllllll llllll ll Two llllllll lllllll lllllllll Three llllllll llll lll l Four lllllllllll llll lllllllll ll

Cut scores The languages exams all have four questions, therefore the highest possible total score is 32. While there is some variation possible in cut-scores, there are limits. For example: 8 must always be N, because it could be gained from N2, N2, N2 and N2. Clearly that’s not an A. Equally, 12 must be an A because it could be gained from A3, A3, A3 and A3 which cannot logically or validly be N or M. 15

Cut score ranges Achievement could start at which four scores? One of the four ‘starting/minimum’ scores is extremely unlikely. –What score is that? –Why is it unlikely? Why must Merit start at 17 and not 16? Which 4 scores could M and E start at? 16