1. Professional Seminar Bucharest 6-10 October 2008 “Assessment: From Programmes to Profiles, Exchanging Best Practices”

2. Overview Background Education and Training Systems Operational focus Drawbacks Common Ground

3. Context In the beginning, when God created all things, he looked upon his creation and said, “It is good.” An Archangel asked, “God, what do you mean by good? What are your criteria? On what data do you base this judgment?”

4. Assessment And thus, on the eighth day, in a blaze of glory, with the archangel trailing flames into oblivion, was the offspring of assessment, the art and science of validation, born.

5. Assessment How do we know when something works? Mostly by noting how it doesn’t.

7. Systems Narrow Context language education and training (most complex of human behaviours) systems approach

8. Analysis Design Development Conduct Evaluation Validation Needs Assessment Typical System

9. Meet MoD Requirements Number of candidates on course Success rates Percentage of requirement met Provide Access to Learning Opportunities Number of accreditation levels Number and location of venues Number of marketing and information tools Percentage of documentation in central repository Program Relevance measure indicator

10. Program Relevance Meet Client and User Expectations Percentage of implementation of graduate feedback Percentage of implementation of results of employer feedback

11. Program Delivery Qualified Instructors Number of qualified professionals Rate of professional development Rate of involvement in outside activities Rate of motivation Rate of competence Percentage of consistency in teacher training Linguistic Aptitude of Students Average entry levels

12. Program Delivery Selection Percentage of qualified nominations Percentage of enrolment procedures advertised Retention Percentage of voluntary terminations Percentage of involuntary terminations

13. Program Coherence Controls Percentage of control documents reflecting needs of MoD Percentage of documents current and complete Percentage of documents meeting system principles Teaching Methods Percentage of application of best practices Rate of integration of student needs Rate of application of measurement tools for assessment of learning

14. Program Coherence Program Delivery Rate of integration of ancillary values Percentage of throughput Percentage of completion in time required Coherent Courses Degree of skill mastered Degree of coherence: learning activities and standards Degree of compatibility: general and specific learning activities Degree of sequence balance in program elements Degree of accuracy of assessment and evaluation tools

15. Program Management Instructional Load Class size Staff/student ratio Percentage of office hours to classroom hours Rate of use of student support services Pedagogical Tools Number of resources available Percentage of resources used

16. Program Management Operating Budget Number of special requests Percentage of accuracy of forecasts Management Rate of integration of staff support services Rate of joint action of program committees and management Rate of integration of program review recommendations

17. Student Assessment Advances in the last twenty years: Research methodology Practical advances Research into performance affect The nature of authentic or performance assessment Ethics of testing

18. Student Assessment Performance assessment Motivation and attitude variables –Integrativeness, attitudes to learning situation, motivation, integrative orientation, and instrumental orientation, age, availability Proficiency Testing Impact on social structures

19. Operations Our collective focus is operational – all assessment must have a demonstrable correspondence between language test performance and language use in this context Is an assessment useful in this context?

20. Drawbacks What happens when assessment does not correspond with learning? Interactivity Spontaneity Communication Exploration Syntax Accord Orthography Formality

21. Common Ground Basic concepts, vocabulary, and procedures are essential for effective communication within our community of practice – i.e., our teachers and testers and managers and students.

22. Learn and Enjoy Bonne route

23. Bucharest October 2008