1. A Tribal Language Program
Development of Tribal Language Arts Standards, Assessments, and Curriculum Maps for
A Tribal Language Program
Presented by:
Charles Stansfield
At CCSSO National Conference on Student Assessment, Minneapolis, June 27, 2012
© 2012 Berlitz Languages, Inc. All rights reserved. 1

2. Progression from BIE contract to Tribal School contract
Agenda
Progression from BIE contract to Tribal School contract
:SLTI provided technical assistance to Tribes in the development of oral native language assessments as part of an alternative definition of AYP.
April 2012: BIE contract ends; Tribal contract begins.
May 2012: SLTI develops oral language assessments, supported by standards and curriculum maps for a tribal school.
© 2012 Berlitz Languages, Inc. All rights reserved.
Second Language Testing, Inc. CCSSO June 27, 2012

3. Considerations Incorporated in Oral Assessments
Agenda
Considerations Incorporated in Oral Assessments
Assess the Spoken Language Only
Honor the Contexts and Scope of Language Use
Reflect Cultural Perspectives, Practices, and Values
Measure Achievement of the TLA Standards, not Language Proficiency
This presentation is a report on our experience developing language proficiency assessments for a Southeastern Native American tribe with a language revitalization program. The tribe has Tribal Language Arts (TLA) standards. SLTI was contracted by the tribe to assist them in revising their TLA standards, in creating curriculum maps linked to the standards, and in developing oral language assessments measure attainment of those standards.
© 2012 Berlitz Languages, Inc. All rights reserved.
Second Language Testing, Inc. CCSSO June 27, 2012

4. History of TLA Standards Development
TLA Standards Revision & Refinement
Tribal Language Arts (TLA) Standards were previously developed, based on the State Standards, for grades K – 12. These included Language, Culture, Science, & Math content.
SLTI staff reviewed and revised the TLA standards in a collaborative process with teachers, administrators, and a member of the Tribal Council.
© 2012 Berlitz Languages, Inc. All rights reserved.
Second Language Testing, Inc. CCSSO June 27, 2012

5. Test Design Elements Test Delivery Only Oral Language is Assessed
Prompts consist of:
Spoken questions
Graphics
Recorded Narrative
Examinees respond orally
Responses may consist of:
Pointing to a visual on TPR (listening)
Single words or phrases
Sentences
Dialogue
The Tribe desires to teach and assess only the spoken language. All responses are either verbal or nonverbal through selection of an image.
Some questions require only a short answer. Others an extended response. Some tasks involve a dialog between the examiner and the examinee.
© 2012 Berlitz Languages, Inc. All rights reserved.
Second Language Testing, Inc. CCSSO June 27, 2012

6. Oral Assessment Structure
Test Delivery
Oral Assessment Structure
Warm –up
Simple questions to get acquainted
Task 1: Listening comprehension: Follow Directions
Prompts consist of oral commands or requests to identify what is shown in graphics
Examinees respond nonverbally
Task 2: Identify Images
Prompt consists of images at first grade, or a recorded narrative at higher grades
Examinees respond with words or phrases
Warm-up. How are you? What’s your name? How old are you?
Follow directions: Using cuisenaire rods, pick up the red stick and give it to me. Put the blue stick in the circle. Put two yellow sticks on the square.
One can see the incorporation of math and science and social studies concepts in the test questions and tasks.
Task 2. Identify images. Manipulatives: Dog, fish, frog, turtle, corn, potato, shoes, etc.
Second Language Testing, Inc. CCSSO June 27, 2012

7. Oral Assessment Structure
Test Delivery
Oral Assessment Structure
Task 3: Describe a picture
Examinee describes objects/people in a picture.
Task 4: Using Pictures, Tell a Story
Examinee speaks about a series of pictures.
Task 5: Short Topical Conversation
Examinee and Examiner discuss a topic
Picture description: kids playing a traditional tribal ball game.
Tell a story: Pictures (waking up, getting dressed, eating breakfast, getting on the bus) showing morning routine of boy. Tell a story about the boy.
Short topical dialog. Possible topics: My favorite animal, teacher, person, etc.
Directions given first in TL, then in English to make sure examinee comprehends.
© 2012 Berlitz Languages, Inc. All rights reserved.
Second Language Testing, Inc. CCSSO June 27, 2012

8. Decisions Influencing Test Structure
Choice of tasks for assessments
Achievement of TLA standards, not proficiency
Skills to be tested
Listening and Speaking
Individual administration
Small class size permits ease of 1-to-1 testing
Taboos
Topics that are taboo were excluded
Each test is audio recorded for later scoring. Video recording used in pilot testing to create administrator training materials.
What are some taboo topics? Certain animals (owl, bear, snakes), death,
© 2012 Berlitz Languages, Inc. All rights reserved.
Second Language Testing, Inc. CCSSO June 27, 2012

9. Decisions Influencing Test Structure
Real-life scenarios
Tasks reflect authentic contexts in which the language is used
Standardizing authentic language in prompts
Orally delivered questions were discussed by TL teachers, then recorded in the TL.
Very important at higher grades.
Training of test administrators includes listening to recordings to ensure consistent, natural delivery of prompts.
Teachers discussed questions in TL and English. Since they could not write down the questions in the TL, they saved them in English. Then they agreed on how the question should be stated in TL. Then, they recorded it on a digital recorder. That is the record and it is reviewed by teachers prior to each admin.
© 2012 Berlitz Languages, Inc. All rights reserved.
Second Language Testing, Inc. CCSSO June 27, 2012

10. Scoring of Assessments
Scoring Rubrics
Development team creating rubrics to reflect goals of instruction
The criteria applied varies by test part.
Evaluation Criteria
Overall Comprehensibility
Task Completion
Pronunciation
Vocabulary
Grammar
Fluency
Fluency may be a criterion when rating performance on higher level tasks at higher grades.
© 2012 Berlitz Languages, Inc. All rights reserved.
Second Language Testing, Inc. CCSSO June 27, 2012

11. Some Things We Have Learned Thus Far
Each tribe has different values, attitudes, and taboos. These must be respected in the test design.
Tests situations must be on the reservation in order to be authentic
The use of photos and drawings helps create a relevant context for many test tasks.
Either or both languages can be used to ensure that test directions are conveyed successfully.
Piloting of tasks and scoring system is very important.
Standardized delivery of spoken language questions is very important.
A final report on the development of the standards and test will be presented to the Tribe. The report will be suitable for delivery to ED to support its inclusion of the TLA test in the tribes definition of annual yearly progress.

12. Next steps Pilot testing (June 2012) Revisions (July 2012)
Field testing and pretesting (August 2012)
Revisions (August/Sept 2012
Final report (Sept 2012)
Operational admin (March 2013)
A final report on the development of the standards and test will be presented to the Tribe. The report will be suitable for delivery to ED to support inclusion of the TLA in the tribe’s proposed definition of annual yearly progress.