1. Standard development in asessment for learning: Strategies and challenges Lars Sigfred Evensen, NTNU Ragnar Thygesen, UiA Gustaf U. Skar, HiST

2.  Adapt to learning environments (cf. Baird et al 2014) =  Aim for ecological validity (Cicourel 1997) = sustainability (Holliday 1994)  Communicate with ‘teacherese’  Specific enough to inform ‘feed forward’ in class  Long-term investment needed to build an interpretive community Some implications by assessment for learning

3.  Develop specific norms of expected proficiency in dialogue with practitioners:  Audiotaped «assess aloud» sessions across the country => locating common criteria formulated in teacherese => discussed in national seminars and refined => used as a basis for target setting and assessment in ordinary classroom work => analyses of outcomes  Ordinal scale with 5 steps (‘levels of mastery’) used across seven sub-domains of writing  The middle step = ‘as to be expected at this year level’ Study 1: The NORMS approach

4.  Non-explicit levels of mastery except mid level  Tacit knowing among teachers instead of metalinguistic awareness, in some domains (but cf. Matre & Solheim in press about development)  Reliability: ICC overview Challenges

5. May 2013January 2014May 2014 Lower grades (3/4).556.618.687 Upper grade (6/7).691.703.702 ICC developments over time (rater pairs, summed scores, one-way random, single measure)

6.  Refining ‘norms of expectation’ on the basis of rated texts (item analysis) to develop specific descriptions of all ‘levels of mastery’  National panel of trained teacher raters  ICC results (comparable to NORMS results) Study 2: A national approach

7. ConsistencyConcensus Spearmans rhoICC Cohens  Total agreementAdjacent agreement Task u1483 (n=62) 0,61 0,4651 %95 % Task u1485 (n=140) 0,680,70,5654 %96 % Reliability indicators across tasks

8.  Not directly linked to classroom practice (as opposed to study 1)  Further refinements needed (where to from here?)  MFRM-approach (sources of variance, scale characteristics)  Standard setting seminars with external domain specialists  Common scale across year levels? (cf. NZ)  Which approach(es) would you suggest? Challenges

9.  Baird, J.-A.,; Newton, P.; Hopfenbeck, T. P.; Steen-Utheim, A. T. & Stobart, G. (2014). Assessment and learning: A state of the field review. Oslo: Norwegian Knowledge Centre for Education.  Cicourel, A. (1997). Ecological validity in pragmatic research. Pragmatics & Cognition, 4, 221-64.   Evensen, Berge & Thygesen (forthc.). Standards as a tool for teaching and assessing cross-curricular writing. The Curriculum Journal.  Holliday, A. (1994). Appropriate methodology and social context. Cambridge: CUP.  Matre, S. & Solheim, R. (in press). Writing education and assessment in Norway: Towards shared understanding, shared language and shared responsibility. L1 Educational Studies in Language and Literature. Special issue on paradoxes and negotiations.  Thygesen, R. & Eggen, A. B. (2015). Bridging summative and formative assessment. In: Cameron, D. L. & Thygesen, R. (eds.), Transitions in special education: Theoretical perspectives and implications for practice. WAXMANN. References