Using Proficiency Data in Working with ALL Stakeholders AUDII Administrators August 3, 2015 Kerrie Neu- Granite Dual Immersion Specialist kneu@graniteschools.org

Objectives Identify the stakeholders in immersion State in parent-friendly language what proficiency is. Tell how to use proficiency data with: Parents Teachers Administrators/Supervisors

Who are the stakeholders? Turn to a neighbor and quickly identify the parties interested in dual immersion and why they are interested. Parents Teachers Students Administrators Legislators Business Partners

Do they feel this way? Questions about student progress Unrealistic expectations Disappointed in perceived results November 2012 Implementing Student Proficiency Reports Proficiency Targets Questions had come up at a school about student progress This problem illustrated that we had not educated parents to understand language acquisition and proficiency targets For me this all began in November of 2012 while driving to a presentation my husband and I were doing. We had begun implementing the student proficiency reports in our district. We were focusing more on the Proficiency Targets and how to assess them Questions had come up at a particular school about student progress. In fact, parents and even some teachers thought that the students were not progressing quickly enough, but these questions were not based on any data. In fact their questions often conflicted themselves, indicating they didn’t understand language acquisition and proficiency targets. While driving and thinking about this problem, I began to reflect how great it would be if parents could speak the same language and understand the same vocabulary as we talk about things like novice, intermediate, creating sentences versus memorized formulas, etc.

The importance of speaking the same language… fluent proficiency novice intermediate advanced low, mid, high

Parents are Organizing Utah Mandarin Immersion Parent Council What information would you like to share? Since then I have learned that Parents are forming their own organizations and creating web sites and Facebooks pages. They share information with each other. It’s important we get the word out so information can be accurate. .org sites, Councils, facebook pages, writing e-books, networks (Minnesota Advocate for Immersion Network) Parent’s Guide to Immersion e-book

Example of Information Shared with Parents Final Grade: A Feedback: “During French Camp, Emilie improved her computer test score by a significant 37%! She also improved her speaking score. Based on her final test scores, she would be placed in a college first-semester French class.” Partner talk: What do you know about Emilie’s language proficiency? What is unknown?

Deductions Known Unknown Emilie performed on the assignments and test that met the teacher’s criteria for an A She improved her computer test score by 37% The improvement is considered significant. She improved her speaking score. Recommended placement in college first-semester French class The criteria for an A is unknown What is the computer test? What was the score? What was her speaking score before and after? Is the recommendation for college first semester French class the same that would be taken by students not having had any French previously?

Student Proficiency Report Partner talk: What do you know about this student’s language proficiency? What is unknown?

Deductions What evidence was used to determine the level? Known Unknown Student performed at Novice Mid level for Speaking Child is performing at grade level Uses words, multiple words, phrases, polite expressions, greetings, & memorized expressions Limited number of topics Frequent searching for words is common Inaccuracies may occur when trying beyond memorized phrases What evidence was used to determine the level?

AAPPL Individual Report This is what the Individual Report looks like. Students take the test in the spring and receive their scores in the fall just like CRT tests. There will be a web site available on the USOE site to help explain the reports. If you look at this sample, you can see the AAPPL score on the left. AAPPL breaks the Novice and Intermediate levels down into numerical levels. You can see this student scored an N4. What I like is that it gives a description of what students can do and then gives some suggestions and strategies to improve the student’s language proficiency. Partner talk: What do you know about this student’s language proficiency? What is unknown?

Deductions Student performed at N4 level for Listening & Speaking Known Unknown Student performed at N4 level for Listening & Speaking Can be part of a conversation about yourself & your life Tend to speak in phrases and some sentences Can often ask and answer simple questions. Is my child performing on target for the grade level?

State Initiative World Language Core focused on Proficiency for K-12 Training for teachers, administrators, and parents

USOE World Language Core For learners, the purpose is to: Provide clear descriptions of what can be done with language at various levels and make expectations more realistic. Offer examples of small, incremental, and achievable goals

USOE World Language Core For teachers, the purpose is to: Guide facilitation of language learning toward more functional, communicative and intercultural goals, rather than those of language structure and cultural fact. Clarify what learners need to be able to do in order to move from one level to the next. 2 of the 5 purposes for teachers

USOE World Language Core For parents, administrators, and other stakeholders, the purpose is to: 1. demonstrate how world language learning has moved from a focus on grammar and translation toward effective communication, literacy, and cultural interaction. 2. demonstrate how the shift has occurred in classical languages from decoding and translation to interpretive reading proficiency. 3. define exactly what is expected of learners at different levels of proficiency. 4. emphasize real-world application for language use.

Training Parents Developed 1 hour presentations for parents with an overview of the Student Proficiency Reports Basic understanding of Proficiency & ACTFL Proficiency Guidelines Prepare them to talk about student progress and assessment data We began to think that it would be great if parents could have similar trainings like our teachers. We knew we couldn’t go in-depth, but at least to give them some understanding and the basic ideas. So we developed 1 hour presentations for parents that helped parents read and understand the Student Proficiency Reports and then have a basic understanding of the ACTFL Proficiency Guidelines and what proficiency is versus achievement or their usual reports of grades. We wanted t o prepare them to have conversations with us about student progress and assessment data.

Overview of Parent Presentation Proficiency Student Proficiency Report ACTFL Proficiency Guidelines for Speaking Proficiency in the real world (What can my child do with the language?) Parental support of students Assessments are a snapshot for that one time Assessment Data In this presentation we talked about What is Proficiency, How to Read the Student Proficiency Report, the ACTFL Proficiency Guidelines for Speaking, What can students do with the language in a real world context. (We found this part to be important as parents felt like “Intermediate” wasn’t good enough until they began to realize what kinds of careers require intermediate or what the students could actually do with an Intermediate level.) Then of course the next piece that came from that was “How do I help my student move from one level to another when I don’t speak the language?” They were used to helping their children with English Reading but didn’t know how to help with Chinese reading for example. We also wanted to emphasize the idea that assessments are just a snapshot of the student’s abilities at that moment in time. We wanted to set the state for writing samples, the AAPPL assessment results, and other proficiency data teachers would share with parents. Keeping the results in this context would be important as students don’t always perform at the same level but move up and down depending on the task and topic.

Why include parents? Parents and teachers talking the same language about student progress Parents understand language acquisition process better Patience for child Encouragement to progress Understanding program goals Understanding of state & classroom assessment results Talk with a neighbor for 30 seconds about how it would help to include parents in understanding proficiency.

Proficiency Presentation for Parents We’re breaking the presentation down in to modules that can be given throughout the year and be put on the website for parents to access on their own. This if the first module: What is proficiency and why do we focus on it? Why do we focus on proficiency? What about grammar, vocabulary, and skills?

What is language proficiency? It’s not about passing a test or getting the right answer. It’s about what students can DO with the language. Is the meaning understood even though everything may not be grammatically correct? In order to understand the proficiency reports, we first have to understand proficiency. What is language proficiency? This is a paradigm shift from when we took language classes or the parents of our students took classes. This is not our students’ mother’s foreign language classes as they say.

Paradigm Shift Shifting from focus primarily on grammar, vocabulary, and skills to a more comprehensive view of proficiency. Proficiency is “what individuals can do with language in terms of speaking, writing, listening, and reading in real-world situations in a spontaneous and non-rehearsed context.” (ACTFL Guidelines 2012) Shifting from focus primarily on grammar, vocabulary, and skills to a more comprehensive view of proficiency. Proficiency is “what individuals can do with language in terms of speaking, writing, listening, and reading in real-world situations in a spontaneous and non-rehearsed context.” (ACTFL Guidelines 2012)

Learning a Language is Like Learning a Sport Takes time to learn Rules, vocabulary, and skills to learn Proficiency is what individuals can do with the sport in real-world situations in a spontaneous and non-rehearsed context. To help our parents understand, we can compare it to learning a sport. They will know that one still needs to learn the rules, vocabulary, skills, etc., but the application of them in a real game is when we see how proficient the player is.

Learning a Language is Like Learning an Instrument Takes time to learn Symbols, skills, vocabulary, and concepts to learn Proficiency is when we can actually use the musical skill to communicate or perform in some way. Another example can be an instrument.

Musical Proficiency? Play this piece. Name the four movements generally associated with a symphony. an opening sonata or allegro a slow movement, such as adagio a minuet or scherzo with trio an allegro, rondo, or sonata Which one demonstrates proficiency better?

Understanding the Student Proficiency Reports This module helps parents understand the reports that we use in Granite District to communicate proficiency in the spring.

Student Proficiency Reports This is a sample of the Student Proficiency Report. It is explained at the Fall SEP Conference and given out at the Spring SEP Conference in grades 1-8. What it is

2013 Proficiency Targets

This is what the Student Proficiency Reports look like This is what the Student Proficiency Reports look like. This one is a Spanish 5th grade report, but the other languages and grade levels are designed similarly.

This is page 2 of the report.

Student Proficiency Report Communicates proficiency, or what the students can do with the language. Follows the national ACTFL standards (American Council of Teachers of Foreign Languages) used across the nation to communicate language proficiency. http://actflproficiencyguidelines2012.org/ Thus the Student Proficiency Reports communicate what the students can do with the language. They communicate proficiency not academic achievement. They follow the national ACTFL standards that describe language proficiency levels. For more information, one can view ACTFL’s web site and see examples of speaking, listening, reading, and writing in the different levels.

ACTFL Proficiency Guidelines Novice Intermediate Advanced Superior Distinguished Listening Speaking Reading Writing The ACTFL Proficiency Guidelines are divided into five levels and 4 areas.

1st grade Area: Listening Shaded box is the target for the grade level. Checked box is your student’s level. On the Student Proficiency Report, it lists the area (listening ability) The target for that grade level is shaded in gray. The teacher will mark the current level of the student with a check in the box.

Let’s start with speaking because if you know what a person can say with the language, you know what kinds of things they can also hear. And actually most people can listen at a higher level than they can speak. The student proficiency report shows four boxes with descriptions in each box of what a student at that level can say in the language. The box that is highlighted in gray is the target level for the end of that grade level.

Speaking Novice Intermediate Advanced Words, memorized phrases or sentences Can create with language, ask and answer simple questions on familiar topics, and handle a simple situation or transaction. Paragraphs and more. Narrate and describe in past, present, and future times/aspect, and handle a complicated situation or transaction. These are what the three levels look like that we’re talking about today. Remember this is just a nutshell view. In reality, there are nuances and subtle differences that require hours of study and practice to be able to fully understand. However, for us to get a general idea, it will help us recognize the stages through which students progress. The first level is Novice. Novice students speak in words, memorized phrases, or memorized sentences. Intermediate speakers can take those memorized phrases and chunks and rearrange them or change them to create new sentences. They can ask and answer simple questions on familiar topics and can handle a simple situation or transaction. Advanced students can speak in paragraphs (if we were to think of the spoken word in the terms of the written word). They can narrate and describe in past, present, and future times. They can handle complicated situations or transactions. We’ll look at them in more details, but this is the easiest way to remember the general ideas. Novices speak in words and memorized chunks, intermediates in sentences, and advanced in paragraphs.

Speaking Novice Intermediate Advanced Parrot Survivor Story teller or reporter Here is another way to look at it given us by Chantal Thompson at a training in 2012. Novices are like parrots. They can repeat back those memorized chunks and phrases. Intermediates are like survivors. They have the basics that they need to survive in a country, but it’s not pretty. They aren’t eloquent and they make many errors. However, they can survive. Advanced students are compared to story tellers. They can describe and narrate in the past, present, and future and speak across many subjects and in many situations. These images can sometimes help us think of the different levels of speakers as we look at our students. From Chantal Thompson

As we learn language, it’s important for us to remember two things. 1. As we progress throughout the levels, it will take more time to advance. This is because we need to be able to understand many subjects and topics. So an advanced speaker must be able to understand vocabulary and language appropriate for many different topics and subjects. It takes time to not only progress vertically through the grammar, but to also gain the breadth of knowledge to converse in many subject areas. 2. We spend a lot more time in the mid ranges of each level than we do the other levels.

Low, Mid, High? Low- Attempts but is unable to sustain. Mid- Solid in this level. Sustainable. High- Solid in current level and actually is trying the next level, but not successfully. The ACTFL levels are further broken down into sub-levels: low, mid, and high.

Shelby County Schools

Oral Proficiency Levels in the Work Field Functions Corresponding Jobs/Professions Examples of who has this level of proficiency Utah Dual Immersion Target Distinguished Ability to tailor language to specific audience, persuade, negotiate. Deal with nuance and subtlety. Diplomat, Contract Negotiator, International Specialist, Translator/Interpreter, Intelligence Specialist Highly articulate, professionally specialized native speakers; L2 learners with extended (17 years) and current professional and/or educational experience in the target culture   Superior Discuss topics extensively, support opinions and hypothesize. Deal with a linguistically unfamiliar situation University Foreign Language Professor, Business Executive, Lawyer, Judge, Financial Advisor Educated native speakers; Educated L2 learners with extended professional and/or educational experience in the target language environment Advanced High Narrate and describe in past, present and future and deal effectively with an unanticipated complication Physician, Military Linguist, Senior Consultant, Human Resources Personnel, Financial Broker, Translation Officer, Marketing Manager, Students with graduate degrees in the language Advanced Mid Fraud Specialist, Account Executive, Court Stenographer/Interpreter, Technical Service Agent, Heritage speakers Informal learners, non-academic learners with who have significant contact with language (LDS Returned Missionaries) Grade 12 Advanced Low Customer service representatives, Police officers, school teachers, Social Worker, Legal Receptionist Undergraduate language majors with year-long study abroad experience LDS Returned Missionaries Grade 10-11  Intermediate High    Create with language, initiate, maintain and bring to a close simple conversations by asking and responding to simple questions Aviation personnel, Missionary, Tour guide, Auto Inspector Undergraduate language majors without year-long study abroad experience After 6 years of middle/high school, AP Grades 8-9 Intermediate Mid Cashier, Sales clerk (highly predictable contexts) After 4 years of middle/high school, AP or 2 years college Grade 6-7 Receptionist, Housekeeping Staff After 2 years of high school Grades 4-5 Intermediate Low Novice High Communicate minimally with formulaic and rote utterances, lists and phrases After 1 year of high school  Grades 2-3 Novice Mid Grade 1 Novice Low This slide helps us see what our students can do with the proficiency levels they gain throughout the program. These are minimum proficiency levels needed for various professions, and how a student typically gains this proficiency. Remember these are typical students. On the right, you see our proficiency targets for Utah Dual Immersion programs. If the students continue with the program, they will graduate from high school with an Advanced Mid, which typically students with graduating from college attain. (Source: http://nble.org/wp-content/uploads/2012/03/ACTFLWorkplaceProficiencyAssess.pdf Dr. Elvira Swender of the American Council on the Teaching of Foreign Languages (ACTFL) CIBER 2012 Conference, March 21, 2012, Chapel Hill, NC)   NOTES: 1. The levels indicated are minimal proficiency levels for specific job descriptions and have been established by subject matter experts from a variety of agencies, organizations and companies for whom ACTFL provides oral proficiency testing following an analysis of the linguistic tasks and the responsibilities of the positions. 2. The references to how long it takes to reach certain levels of proficiency were written specifically for the study of Spanish, a Category I language. Other Category I languages include Afrikaans, Danish, Dutch, French, Haitian Creole, Italian, Norwegian, Portuguese, Romanian, Swahili and Swedish. For Category II, III and IV languages, one can expect that it will take longer to reach the same levels of proficiency. Dr. Ray Clifford of the BYU Language Center says RM’s test about Intermediate Mid to Intermediate High. Chantal Thompson updated info on LDS Missionaries to between Intermediate High to Advanced Low or Advanced Mid) From Dr. Elvira Swender, ACTFL, CIBER 2012 Conference

Speaking- Novice Low -Uses isolated words (i.e., single words) to respond to questions. -Responses pertain to very specific topic areas in predictable contexts. - May use greetings and polite expressions such as Good Morning and Thank you. Let’s look at Speaking now in detail.

Speaking- Novice Mid 1st -Uses single words, multiple words, short phrases, greetings, polite expressions, and other memorized expressions on a limited number of topics. -Frequent searching for words is common. -May use native language or gestures when attempting to create with language beyond what is known. -Memorized expressions with verbs and other short phrases are usually accurate, but inaccuracies occur when trying to produce language beyond the scope of memorized material. Remember this is our target for 1st grade

Speaking- Novice High 2nd & 3rd Partial ability to create with language to convey personal meaning by adapting learned material in single sentences and strings of sentences ask and answer questions handle a simple survival situation (daily needs) in the language Uses vocabulary from everyday topics and subject area content to provide basic information. Uses memorized expressions with ease and accuracy. Can respond in intelligible sentences most of the time but does not sustain sentence-level speech Sentences may not always contain the proper verb formations, and other grammatical inaccuracies may be present. May revert to the use of English when foreign language words cannot be retrieved or when dealing with unfamiliar topics.

Speaking- Intermediate Low 4th & 5th Sustained but minimal ability to create with language to convey personal meaning by adapting learned material in single sentences and strings of sentences ask and answer questions handle a simple survival situation (daily needs) in the language Has basic vocabulary for making statements and asking questions to satisfy basic social and academic needs, but not for explaining or elaborating on them. Can maintain simple conversations at the sentence level by creating with the language, although in a restrictive and reactive manner. Handles a limited number of everyday social and subject content interactions. Uses a variety of common verbs in present tense (formations may be inaccurate) Other verb tenses/forms may appear but are not frequent. The listener may be confused by this speech due to the many grammatical inaccuracies.

Analogy This analogy helps us understand why we often see more errors in their speaking when they read the Intermediate stage. Think of a baby who can crawl very proficiently. When they first begin to stand and walk, they fall down a lot and are not solid in the skills of walking. Yet, we don’t tell them to go back to crawling because they made less errors then. We encourage them to try the next level of walking until they become skillful at it. In the same sense, Intermediate students are attempting a higher level of speech. They make more errors at first until it becomes more comfortable for them. As parents and educators, we encourage the students to attempt higher levels and reassure them when they make errors.

Speaking- Intermediate Mid 6th & 7th Confident ability to create with language to convey personal meaning by adapting learned material in single sentences and strings of sentences ask and answer questions handle a simple survival situation (daily needs) in the language   Has basic vocabulary to permit discussions of a personal nature and subject area topics. May attempt circumlocution when appropriate vocabulary is missing. Maintains simple sentence-level conversations. May initiate talk spontaneously without relying on questions or prompts. May attempt longer, more complex sentences, including the use of basic sentence connectors (e.g., and, but, however) Uses an increasing number and variety of verbs. Verbs are mostly in present tense although awareness of other verb tenses (future/past) and forms may be evident. Meaning is generally clear in spite of some grammatical inaccuracies.

Speaking- Intermediate High 8th & 9th Partial ability to converse freely on autobiographical topics as well as issues related to daily living (in school, home, community) describe and narrate across the major time-frames of present, past and future speak in paragraph-length utterances have good control of basic structures and vocabulary to be understood without difficulty by native speakers, including those unaccustomed to language learners Has a broad enough vocabulary for discussing simple social and academic topics in generalities, but may lack detail. Sometimes achieves successful circumlocution when precise word is lacking. Initiates and sustains conversations by using language creatively. Shows a developing but not sustained ability to use paragraph-level speech with connected sentences (e.g., then, so, that, etc.) in descriptions and narrations Control of present tense is solid but patterns of breakdown appear in past and future timeframes Grammatical inaccuracies are still present.

Determining proficiency level over time Proficiency level not achievement or performance Is the overall meaning understandable? This is where they are for that topic and task. Students may demonstrate different proficiency levels. Over time look at the general trends. It is very important to keep in mind that a writing sample is just one snapshot of the student’s writing abilities THAT day on THAT topic. Proficiency levels are determined over time with multiple samples. So we look at their results over time and see what general proficiency level they demonstrate in their writing. Looking at this chart, you can see the student generally began in the Novice Low area and by May was writing in the Novice Mid level. Some samples are higher, some may be lower, but this is a general range.

Assessing Language Competencies - STARTALK Session 5/4/2013 Islands of performance merge into continents on the way to global proficiency! As students learn a piece of language, it adds to their overall proficiency picture. We like to think of these are islands of proficiency. For example, a student could learn the basic greetings in that language. That is an island so to speak. Later they learn how to tell about themselves giving personal information. That is another island of proficiency. Once we combine many islands of proficiency, we build continents of proficiency. Eventually, as we add more language skills, vocabulary, and experience, those continents of proficiency build global proficiency.

AAPPL Measure ACTFL Assessment of Performance toward Proficiency in Languages Addresses the National Standards for Foreign Language Learning Assesses following modes of communication: Interpersonal Listening/Speaking- Grades 3, 5,6,7,8 Interpretive Reading- Grades 4 6,7,8 Presentational Writing- Grades 6,7,8 The AAPPL Measure is the ACTFL Assessment of Performance toward Proficiency in Languages. It addresses the National Standards for Foreign Language Learning. In Utah we assess the following modes of communication (see slide)

Performance Vs. Proficiency Demonstrate the features of the range in certain contexts and content areas that have been learned and practiced. Demonstrate consistent patterns of all of the criteria for a given level all of the time. One thing to keep in mind is that the AAPPL test does not give us a definitive proficiency level. That would require more in-depth assessment. What it does give us is a performance assessment that gives us a good indication of the progress being made towards a proficiency level. It demonstrates their abilities in certain contexts and content areas that have been learned and practiced. Remember that proficiency demonstrates consistent patterns of all of the criteria for a given level all of the time. Still this is helpful for us to see how the students are doing and gauge their progress. It is a standardized test given on a computer and rated by national scorers.

AAPPL Individual Report This is what the Individual Report looks like. Students take the test in the spring and receive their scores in the fall just like CRT tests. There will be a web site available on the USOE site to help explain the reports. If you look at this sample, you can see the AAPPL score on the left. AAPPL breaks the Novice and Intermediate levels down into numerical levels. You can see this student scored an N4. What I like is that it gives a description of what students can do and then gives some suggestions and strategies to improve the student’s language proficiency.

What Parents Can Do Overview Support first language Be positive and encourage during difficult times. Follow up with school work and homework. Encourage use of the language whenever possible. Strengthen the connection between home and school.

Using Proficiency Data with Students

Give Students Specific Feedback Proficiency levels & rubrics Can-Do types of statements Proficiency based grading Leading toward Student Proficiency Report

Goals My goal By when Action steps Who will help me? Date Based on WIG: Wildly Important Goals

Getting kids to “level up” Take risks Understand the next level Encourage Mario Ice Cream Scoops Now that you know a bit about the different proficiency levels, you can help encourage your students to level up. We can use a couple of analogies to help the students do this.

Level Up Level up like Mario or video games. “That was a good answer. Now level up and give me a better answer.” Some schools use the Mario “level up” idea. Students know they have to try a new level until they succeed in order to win the game. When they give us a simple answer, we can encourage them to “level up” and give me a more complex answer. Perhaps it’s more words or a sentence instead of just words.

Ice cream scoops Tell them to fill up their Ice cream cones with more ice cream. More scoops!! Another example is the image of ice cream scoops. The inverted triangle graph for proficiency resembles an ice cream cone. So we tell students we want them to fill up their cones with more ice cream scoops. We can say, that was a one scoop answer. Can you give me a two scoop answer?

Preparing for the AAPPL Topics Demos Embed social language in curriculum maps and lesson plans Formative assessments

Using Proficiency Data with Teachers Classroom observations Coaching PLC’s

Data to Drive Instruction High listening scores but low speaking scores? Collect data about who does the talking in the classroom. Teacher Individuals Pairs Whole Class

How else can data drive instruction? Low listening scores? Students may need more practice listening to speakers other than the teacher such as videos, audio books, other students, etc. Low writing scores? Additional writing experiences Daily journal, quick-writes, writing process, instruction on how to move to next level.

Using Proficiency Data with Supervisors Show general program performance Garner support from above Demonstrate interest in program

Let’s Review ALL stakeholders need information We want to be the source for correct information We want to speak the same language We want students to understand their own progress. We want to use data to drive instruction

Questions

Contact Information Kerrie Neu kneu@graniteschools.org Handouts available at www.kerrieneu.weebly.com