Professional Seminar Bucharest 6-10 October 2008 “Assessment: From Programmes to Profiles, Exchanging Best Practices”

Overview Background Education and Training Systems Operational focus Drawbacks Common Ground

Context In the beginning, when God created all things, he looked upon his creation and said, “It is good.” An Archangel asked, “God, what do you mean by good? What are your criteria? On what data do you base this judgment?”

Assessment And thus, on the eighth day, in a blaze of glory, with the archangel trailing flames into oblivion, was the offspring of assessment, the art and science of validation, born.

Assessment How do we know when something works? Mostly by noting how it doesn’t.

Systems Narrow Context language education and training (most complex of human behaviours) systems approach

Analysis Design Development Conduct Evaluation Validation Needs Assessment Typical System

Meet MoD Requirements Number of candidates on course Success rates Percentage of requirement met Provide Access to Learning Opportunities Number of accreditation levels Number and location of venues Number of marketing and information tools Percentage of documentation in central repository Program Relevance measure indicator

Program Relevance Meet Client and User Expectations Percentage of implementation of graduate feedback Percentage of implementation of results of employer feedback

Program Delivery Qualified Instructors Number of qualified professionals Rate of professional development Rate of involvement in outside activities Rate of motivation Rate of competence Percentage of consistency in teacher training Linguistic Aptitude of Students Average entry levels

Program Delivery Selection Percentage of qualified nominations Percentage of enrolment procedures advertised Retention Percentage of voluntary terminations Percentage of involuntary terminations

Program Coherence Controls Percentage of control documents reflecting needs of MoD Percentage of documents current and complete Percentage of documents meeting system principles Teaching Methods Percentage of application of best practices Rate of integration of student needs Rate of application of measurement tools for assessment of learning

Program Coherence Program Delivery Rate of integration of ancillary values Percentage of throughput Percentage of completion in time required Coherent Courses Degree of skill mastered Degree of coherence: learning activities and standards Degree of compatibility: general and specific learning activities Degree of sequence balance in program elements Degree of accuracy of assessment and evaluation tools

Program Management Instructional Load Class size Staff/student ratio Percentage of office hours to classroom hours Rate of use of student support services Pedagogical Tools Number of resources available Percentage of resources used

Program Management Operating Budget Number of special requests Percentage of accuracy of forecasts Management Rate of integration of staff support services Rate of joint action of program committees and management Rate of integration of program review recommendations

Student Assessment Advances in the last twenty years: Research methodology Practical advances Research into performance affect The nature of authentic or performance assessment Ethics of testing

Student Assessment Performance assessment Motivation and attitude variables –Integrativeness, attitudes to learning situation, motivation, integrative orientation, and instrumental orientation, age, availability Proficiency Testing Impact on social structures

Operations Our collective focus is operational – all assessment must have a demonstrable correspondence between language test performance and language use in this context Is an assessment useful in this context?

Drawbacks What happens when assessment does not correspond with learning? Interactivity Spontaneity Communication Exploration Syntax Accord Orthography Formality

Common Ground Basic concepts, vocabulary, and procedures are essential for effective communication within our community of practice – i.e., our teachers and testers and managers and students.

Learn and Enjoy Bonne route

Bucharest October 2008