1. Key Issues for Educational Assessment in Europe Eduardo Cascallar and Christina Wikström, AEA-Europe Council Members Kiev, Ukraine – July 2007

2. Overview Standards with different purposes Current issues in assessment research and development Trust in the Assessment Process Fairness for all Participants in the Process Appropriate Use of Information Technology in Assessment Adequate Measurement in Performance Assessments

3. Standards in assessment Standards are ”hot” in educational assessment today, but: The term 'standard' is often used in ambiguous and confusing ways Also in assessment theory, ”standards” is used in many contexts: Standard-based assessment, Standardized assessment, Standard setting etc. Formally, a standard is ”something considered by an authority or by general consent as a basis of comparison; an approved model” (could be anything that meets this criteria)

4. Three types of standards in assessment systems in Europe 1) Focus is internal (standards-based assessment) 2) All conditions are common (Standardization) 3) Focus is external: format and administration

5. 1) Standards-based assessment Standards-based assessment is more formally known as criterion-referenced assessment/measurement. The idea is to measure what the student can do, not how other students are doing (relative/norm-referenced assessment). The criterion is of central importance, often linked to the educational curriculum. Popular (and necessary) for formative assessment, but also for summative and formative purposes – value added measurement, often related to (school) accountability Examples of countries with recent reforms with such purposes in mind; USA (the No Child Left Behind-Act), UK (as discussed by Chris Whetton), Sweden and Norway (new grading system).

6. 2) Standardized assessment Standardized assessment has existed for a long time (roots in psychological testing), and is becoming more common in certain areas, for instance in university admission and in certification/licensing. Admissions are becoming more school oriented – changing approach from the psychometric to being more ”edumetric”. More diversified sub-tests (subject oriented). Licensing/certification tests more directed towards performance assessment, assessment of skills and higher cognitive aspects rather than basic knowledge. A standardized test is designed in such a way that the "questions, conditions for administering, scoring procedures, and interpretations are consistent” and are "administered and scored in a predetermined, standard manner" (Popham, 1999)

7. 3) External Standards (without content) The third type of evolving standard-related assessment program in Europe has no criterion and does not meet the requirements for a standardized instrument. The standard is limited to format and administration, will not contribute to high reliability and validity (internal variations). Example: ECTS (the European Credit Transfer System) Purpose: comparability within/across European countries, transparency. Format: Common grading scale A-F, norm referenced but with additional ”Diploma Supplement” to serve as criterion.

8. ECTS – an international standard? ECTS Transfer system one year full time studies equals 60 credits. ECTS Grading system Norm-referenced Scale: A-F: A (top 10%), B (next 25%), C (next 30%), D (next 25%), FX (FX means “fail- some more work required to pass” and F means “fail – considerable further work required). The purpose of the Diploma Supplement is to provide a standardized description of the nature, level, context, content and status of the studies. The idea is that the DS will provide transparency and facilitate academic and professional recognition of qualifications.

9. ECTS (Cont’d) The ECTS Grading system is now on its way to be implemented in a number of European universities. Relevant questions are: Is the diploma supplement a criterion? What type of assessment is the ECTS? Is ECTS a standard?

10. Issues for research and discussion 1. How to achieve comparability in standards-based assessment within and across schools, regions, countries – same criterion, different teachers, schools, raters, systems etc. 2. How to ensure equity when different student groups are included (especially problematic with multi-ethnicity) 3. How to measure achievement over time, especially when tests and test scores are public – what instruments to use, and statistical techniques 4. Validity aspects – how evaluate the validity of these systems? 5. Do programs like the ECTS have a chance in Europe – and what will the consequences be? ”When you want to measure change, do not change the measure” (R. Zwick)

11. “A true education requires far more than prepackaged tests and a box of No. 2 pencils.” (Hall, 2004) Jacquelyn Hall President Organization of American Historians.

12. Some identified action lines in European education: Adoption of a system of comparable degrees Promotion of mobility Promotion of European cooperation in quality assurance Lifelong learning Promotion of the European dimension (particularly in higher education)

13. Important Issues to Achieve Goals in Educational Assessment: Trust in the Assessment Process Fairness for all Participants in the Process Appropriate Use of Information Technology in Assessment Adequate Measurement in Performance Assessments (including ‘Soft Skills’ and Values: such as team working, problem solving, etc)

14. Trust in the Assessment Process Has to represent the interests of all stakeholders (examinees, teachers, school-district-national authorities, society at large, etc.) Has to represent the values and educational traditions of each society. Must be perceived as fair to all participants.

15. Fairness and Equity Must be real, not just the perception of. Has to take into account differences in culture, language, disabilities, and other issues unrelated to the constructs being measured. Must have quality standards for the instruments used, including levels of reliability, validity, relevance, control of bias, impact.

16. Use of Information Technology Appropriate for the assessment tasks. Must take into account issues of fairness. Should avoid introducing extraneous factors in the assessment. In general: “use the lowest technology necessary for the assessment task, not the most novel or ‘trendy’.”

17. Use of Information Technology: Some new possibilities New skills can be measured adequately:  speaking proficiency in languages,  complex reasoning tasks,  fine motor skills,  problem solving processes Use of new methods, like predictive technologies, to diagnose and classify (assessments beyond testing) with the use of intelligent systems.

18. Performance Assessments: some initial considerations Are they desirable in the assessments? What are the prerequisites needed to guarantee their quality and applicability? When should they be used? What are the consequences of their use (budgetary, impact, biases, etc.)? Is the infrastructure available for their implementation? Do they guarantee validity, reliability and fairness?

19. Quality Considerations for European Educ. Assessments Learn from mistakes in Europe and elsewhere. Consider the political and policy implications of accountability systems. Prepare citizenry to the needs of a changing and interconnected world. Adequate funding of national educational priorities, including assessment to improve the learning process.

20. Looking to the Future of European Educational Assessment: Should be in line with the desired value system and model of citizenship adopted by each society. Should promote common European standards to facilitate mobility Should include systems of accountability and quality assurance

21. Thank you! Christina Wikström Christina.wikstrom@edmeas.umu.se Eduardo Cascallar cascallar@msn.com www.aea-europe.net