1. Assessment in the Less Commonly Taught Languages: Challenges and Approaches William P. Rivers, Ph.D. Executive Director Joint National Committee for Languages National Council for Languages and International Studies September 23, 2013

2. Overview Operational Definition of LCTLs – LCTL vs. “Critical Language” – Common characteristics of LCTL fields Basic types of language assessment – Achievement – Proficiency Why FL has an edge in assessment: Proficiency Challenges in proficiency testing in the LCTLs Alternative approaches About JNCL-NCLIS Q & A

3. What is a “Less Commonly Taught Language?” MLA enrollment survey (Furman et al., 2009): – 1.6m FL enrollments in US IHEs in 244 FLs – 8.4 enrollments per 100 students (essentially unchanged since 1974) – 1.2m of these (80%): Spanish, French, German In effect, all other FLs are LCTLs – 1 st tier: 10,000 – 80,000 enrollments: Chinese, Italian, Japanese, Russian, Portuguese, Hebrew, Japanese – 2 nd tier: 1,000 – 10,000 enrollments: 9 – 3 rd tier: <1,000: 235 FLs

4. Common characteristics of LCTL fields: Capacity challenges Foundational: – linguistic descriptions may be lacking; SLA research in the TL may be lacking – Few or no Ph.D. programs to train future faculty & researchers – Few or no programs to train K-12 teachers Infrastructure: – fewer and less-widely distributed disciplinary journals – Smaller or joint conferences – Few professional development programs Programs – Fewer programs (of course) and – Programs with no advanced level programming – Programs under constant threat of elimination due to low enrollments

5. LCTL vs. “Critical Language” Each USG agency has its own criteria for determining which languages are critical Not all LCTLs are critical Not all CTLs aren’t critical For external funding programs, determination of “critical language” rests on – Agency needs – Programmatic priorities – funding

6. Basic types of FL Assessment Achievement: Discrete point testing of specific items – Typically tied to a specific course or curriculum Proficiency: graduated, structured, progressive testing of language competencies – Curriculum and program independent – Fundamentally communicative Proficiency testing began at Foreign Service Institute in the 1950s – developed by John Carroll and his students – Based on three years’ observation of language use by diplomats – Divided into 4 skills (L/R/S/W) and five levels – This became the basis for the ILR Proficiency Scale – In turn the basis for the ACTFL proficiency Scale (and most others worldwide use the same approach)

7. Proficiency Scale Review ACTFL: – Novice – Intermediate – Advanced – Superior ILR: 0 -5 Professional language skill generally requires the “Superior” or ILR 3 level 4 years of College FL results in a median proficiency of Intermediate High /1 +

8. Why we have an edge FL proficiency testing has revolutionized FL pedagogy in the past 30 years: we now focus on communicative ability as the fundamental goal of FL education Among the Humanities, FL has a unique capability to show what students can do with their knowledge, because proficiency as a construct is precisely that – what can an individual do with their FL 50+ years of practice and R&D in FL testing

9. LCTL Assessment Challenges Range of languages – 240 + languages Geographic dispersion of examinees Low number of examinees – Economic viability of test development – Technical challenges Validation Test design

10. An Incomplete Set of Available Tests: Oral Proficiency Interviews Afrikaans, Akan-Twi, Albanian, Algerian, Amharic, Arabic, Armenian, Azerbaijani, Baluchi, Bengali, Bosnian, Bulgarian, Burmese, Cambodian, Cantonese, Cebuano, Chavacano, Czech, Dari, Dutch, Egyptian, English, French, Ga, Georgian, German, Greek (Modern), Gujarati, Haitian Creole, Hausa, Hebrew, Hiligaynon, Hindi, Hmong- Mong, Hungarian, Igbo, Ilocano, Indonesian, Italian, Iraqi, Japanese, Javanese, Jordanian, Kashmiri, Kazakh, Kikongo-Kongo, Korean, Krio, Kurdish Kurmanji, Kurdish Sorani, Lao, Lebanese, Levantine, Libyan, Lingala, Malay, Malayalam, Mandarin, Mandingo-Bambara, Moro, Nepali, Pashto, Persian-Farsi, Polish, Portuguese, Punjabi, Romanian, Russian, Serbian/Croatian, Sindhi, Sinhalese, Slovak, Somali, Spanish, Swahili, Tagalog, Tajik, Tamil, Tausug, Telugu, Thai, Tigrinya, Turkish, Turkmen, Uighur, Urdu, Uzbek, Vietnamese, Wolof, Wu, Yoruba Source: Language Testing International

11. Challenge: volume Approach: Self-Assessments 4 skill (L/R/S/W) Based on ILR scale Global statements of proficiency Detailed assessments by task and proficiency level (Can-do statements) Regular monitoring of concurrent validity worth formal proficiency tests

12. Self-Assessments: Concurrent Validity 4 skill (L/R/S/W) Based on ILR scale Global statements of proficiency Detailed assessments by task and proficiency level (Can-do statements) Regular monitoring of concurrent validity with formal proficiency tests – Peer reviewed (Stansfield, Gao, Rivers, 2010)

13. Self-Assessments: Concurrent Validity Moderate correlations; acceptable for membership screening

14. Challenge: Range of Languages Approach: Expand access to quality tests ASTM F2889 – 11: National industry standard for proficiency test development Users and providers develop consensus Standard to ASTM format ASTM: American Society for Testing and Materials – 30000+ industry standards – Develops but does not enforce standard Equal representation of users & providers Participation is open & voluntary ASTM F43 on Language Services and Products – 120 members from industry, academe, government – 4 standards developed – 6 underway

15. Value of Standard Practice Leverage – world-leading foreign language proficiency testing (the DLPT system) and practices (the FILR Proficiency Guidelines) to ensure quality procurement of test development and administration – the $15b language services industry’s potential to provide language proficiency testing in more than 150 languages Enable rapid development of foreign language proficiency tests to meet surge requirements Enhance compliance with – PL 104-113 §12(d), which requires US Government agencies to maximize the use of industry standards – DoD Standards Program, DoD 4120.24-M Improve quality assurance through the availability of such standards to procurement and contracting for language proficiency testing

16. Scope of ASTM F2889-11, Standard Practice for Assessing Language Proficiency Establishes specific requirements for quality and transparency in – Test specification – Test development – Test administration Incorporates current best practices – FILR Guidelines – DLPT-V testing – Other USG and industry leading test practices All levels/all modalities are covered

17. Scope of Standard Practice (2) Incorporates current best practices – FILR Guidelines – DLPT-V testing – Other USG and industry leading test practices Mandates specific practices, e.g., – Test frameworks – Ethics – Quality control – Quality assurance

18. The Joint National Committee for Languages and the National Committee for Languages and International Studies Comprise more than 80 organizations in the language enterprise Develop and advocate policies for language and global expertise Promote language as a profession in the United States Support the development of academic, governmental, and industrial standards for language work Have joined with the language industry and the Globalization and Localization Association to form the American Language Enterprise Advocacy (ALEA)

19. Contact Dr. Bill Rivers Executive Director, JNCL-NCLIS Chair, ASTM Main Committee F43, Language Services and Products wrivers@languagepolicy.org