Military Language Testing at the National Defence University and the Common European Framework BILC CONFERENCE BUDAPEST

General language examination in five languages General language examination in five languages ARMA bilingual military examination in nine languages ARMA bilingual military examination in nine languages NATO STANAG 6001 monolingual language examination NATO STANAG 6001 monolingual language examination

 Interviews with officers and NCOs Hungarian officers and NCOs serving at the NATO headquarters Hungarian officers and NCOs serving at the NATO headquarters Job descriptions of soldiers serving in NATO positions Job descriptions of soldiers serving in NATO positions

OOOObservance of the regulations for the conduct of examinations; CCCContinuous training and observing of testers CCCContinuous pretesting and renewal of items;

COMMON EUROPEAN FRAMEWORK

ESL Exams in Hungary University Of Cambridge ESOL University Of Cambridge ESOL Trinity College London ESOL & GESE Trinity College London ESOL & GESE The European Language Certificates The European Language Certificates Pitman Qualifications Pitman Qualifications Österreichisches Sprachdiplom Deutsch ÖSD Österreichisches Sprachdiplom Deutsch ÖSD Goethe-Institut vizsgarendszere Goethe-Institut vizsgarendszere LEXINFO LEXINFO LCCIEB EFB angol üzleti szaknyelv LCCIEB EFB angol üzleti szaknyelv Euro Euro ECL nyelvvizsga ECL nyelvvizsga Bibliai héber Bibliai héber Diplomáciai szaknyelvi vizsga Diplomáciai szaknyelvi vizsga Certifications de l´ Alliance française de Paris Certifications de l´ Alliance française de Paris

Linking Separate Assessments Equating Equating Calibrating Calibrating Statistical moderation Statistical moderation Benchmarking Benchmarking Social moderation Social moderation

Linking Process Selection of appropriate procedures Selection of appropriate procedures Project planning Project planning Coordination of the participants Coordination of the participants Application of the procedures Application of the procedures Recording of results Recording of results

Linking Procedures Familiarisation Familiarisation Specification Specification Standardisation Standardisation Empirical Validation Empirical Validation

Familiarisation Discussion of the Common European Framework levels Discussion of the Common European Framework levels Qualitative analysis of the Common European Framework levels Qualitative analysis of the Common European Framework levels

The Specification Stage

Manual: Relating Language Examinations to the Common European Framework of Reference for Languages: Learning, Teaching, Assessment (CEF)

Specification: Description of the coverage of the examination in relation to the Common European Framework

Content analysis procedures: filling in a number of checklists filling in a number of checklists using relevant CEF descriptors to relate the exam to the categories and levels of the CEF using relevant CEF descriptors to relate the exam to the categories and levels of the CEF

The Aims of the Specification To analyse the content in relation to the relevant categories of the CEF To analyse the content in relation to the relevant categories of the CEF To profile the examination in relation to the relevant descriptor scales of the CEF To profile the examination in relation to the relevant descriptor scales of the CEF To make a claim that an examination is related to a particular level of the CEF To make a claim that an examination is related to a particular level of the CEF

Phase 1: Phase 1: A general description of the examination or test tasks Phase 2: Phase 2: A detailed description of the examination or test tasks A detailed description of the examination or test tasks

Stage 1 General Examination Description Form (A1) What is the overall aim? What are the more specific objectives? What are the principal domains? Which communicative activities are tested? What types of test tasks are used? What information is published for candidates and teachers?

Stage 2 Forms on Different of the Examination Stage 2 Forms on Different Aspects of the Examination The test development process (Form A2) The test development process (Form A2) The marking schemes and scoring rules for different subtests (Form A3) The marking schemes and scoring rules for different subtests (Form A3) The grading and standard-setting procedures for different subtests (Form A4) The grading and standard-setting procedures for different subtests (Form A4) The reporting of results (Form A5) The reporting of results (Form A5) The analysis and review procedures (Form A6) The analysis and review procedures (Form A6)

Form 3A: Marking How are the tests marked? Where are the tests marked? What criteria are used to select the markers? How is accuracy of marking promoted? Are productive or integrated test tasks single or double rated? If double rated, what procedures are used when differences between raters occur?

The Detailed Description of the Examination

Forms for the Communicative Language Activities Listening Comprehension Listening Comprehension Reading Comprehension Reading Comprehension Spoken Interaction Spoken Interaction Written Interaction Written Interaction Spoken Production Spoken Production Written Production Written Production Integrated Skill Combinations Integrated Skill Combinations Integrated Skills Integrated Skills Spoken Mediation Spoken Mediation Written Mediation Written Mediation

Listening Comprehension Which situations, content categories, domains are the test takers expected to show ability in? Which communication themes are the test takers expected to be able to handle? Which communicative tasks are the test takers expected to be able to handle? What kind of communicative activities and strategies are the test takers expected to be able to handle?

After reading the scale for Overall Listening Comprehension, indicate and justify at which level(s) of the scale the subtest should be situated

Forms for the Aspects of Communicative Language Competence  Reception  Interaction  Production  Mediation

Outcome Outcome A narrative description of the examination A narrative description of the examination Completed versions of the relevant forms Completed versions of the relevant forms A graphical presentation of the relationship of the examination to the CEF levels A graphical presentation of the relationship of the examination to the CEF levels

Thank you for your attention