1. STANAG 6001 Speaking Test Observations; Lessons learned
Tamar Shavlakadze
BILC Professional Seminar
Tbilisi, Georgia
1-5 October 2017

2. General Outline of the Presentation
Speaking test format (the core of the test and tester stance)
Global tasks and communicative functions (macro and micro functions)
Evaluation (Fluency & Accuracy)

3. Who are the test takers? Resolute Support Mission;
Promotion to higher ranks;
International positions abroad;
Courses abroad;
Graduates of English language courses of the Language Training School of NDA;
Graduates of Bachelors Programme of National Defence Academy;
Applicants for the positions of teachers and translators.

4. General Test Format Bi-level (levels 1-2; 2-3)
Length: Bi-level minutes
Bi-level minutes
Two examiners (interlocutor/rater)
Guided interview
Three mandatory phases

5. Core of the test and three Perspectives
Level Checks
Probes
Three Perspectives
The core
Psychological
Proves the examinee what she or he can do
Proves the examinee what he or she cannot do
Linguistic
Checks for tasks and content which examinee performs with greatest accuracy
Checks for tasks and content which examinee performs with least accuracy
Evaluative
Finds the examinee’s speaking level
Finds level at which examinees can no longer speak accurately

6. Tester stance
TTT (Teachers/testers) talking time must be at a minimum in both cases;
Lessons as well as speaking tests should be contextualized;
Instruction checking questions should be used;
Teachers/testers should create friendly atmosphere;
Teachers/testers should be in control of the lesson/exam;
Pursue topics with follow-up questions.

7. Tester Stance Testers Should NOT: Correct examinees;
Finish their sentence;
Use body language;
Feed vocabulary item;
Go informal or start chatting with candidates.

8. Global Tasks and Communicative Functions
Macro function – categories for the functional use of spoken and written discourse consisting of a sequence of sentences.
Factually oriented talk
Evaluative talk
description
explanation
narration
justification
instruction
prediction
comparison
decision
Bygate (1987)

9. Micro functions are related to individual actions completed within a turn in an interaction.
CEF groups them into 6 main categories:
giving and asking for factual information, e.g. identifying, reporting, correcting, asking, answering;
expressing and asking about attitudes, e.g. agreement/disagreement, knowledge/ignorance, ability, intentions, preference, emotions, apologies, sympathy.
suasion, e.g. suggestions, requesting, warnings, advice, asking help, invitations, offers;
socializing, e.g. attracting attention, addressing, greetings, introduction;;
structuring discourse, e.g. opening, turn taking, closing, changing the theme;
communication repair, e.g. signaling non-understanding, appealing for assistance, paraphrasing.

10. Global tasks and Communicative Functions
NO PROFICIENCY
Level 1
SURVIVAL
Short conversation
Ask and answer simple questions (present tense)
Handle a simple transaction (RP)
Short Narration in present
Short description
Level 2
FUNCTIONAL
Narration ( Present, Past, future)
Description (person, place, object)
Report/state facts on a current event
Instructions /directions
Compare/contrast
Information Gathering Task (in past tense)
Deal with a familiar situation with a complication (RP)
Level 3
PROFESSIONAL
Supported opinions
Hypothesize
Discussing an abstract topic
State and defend policy

11. MEMORIZED PROFICIENCY
Holistic Assessment Scale
Levels & Tasks
Accuracy Statements
Text produced
NO PROFICIENCY
No functional ability
None or occasional isolated words
Random words & phrases 0+
MEMORIZED PROFICIENCY
Three level checks at level 1
Can ask questions or make statements only with memorized material and set expressions
Most utterances are telegraphic
Linking words and markers are omitted, confused, or distorted
Communication is severely limited
Faulty pronunciation, stress and intonation
Memorized words and short phrases 1
SURVIVAL
Short conversation
Ask and answer simple questions
Handle a simple transaction (RP)
Short Narration
Short Description
Can create sentences
Can begin, maintain, and close short conversations
Can satisfy simple, predictable, personal and accommodation needs
Can meet minimum courtesy, introduction, and identification requirements
Communicate about routine tasks in the workplace
Seldom speaks with natural fluency, and cannot produce continuous discourse, except with rehearsed material
Can speak at the sentence level and may produce strings of two or more simple sentences joined by common linking words
Frequent errors in pronunciation, vocabulary, and grammar often distort meaning
May only use one tense or tend to avoid certain structures
Speech is often characterized by hesitations, erratic word order, groping for words
Ineffective reformation and self-corrections
Discrete sentences 1+
SURVIVAL+
Three probes at Level 2 (RP)
Can easily take part in short conversations by asking and answering simple questions
Shows a very limited and inconsistent ability to handle longer conversations on concrete topics
Discourse consists of strings of related sentences but not full paragraphs
Can satisfy a few social demands and provide more than skeletal information when supplying biographical background
Can readily ask for assistance; request information and clarification; and express satisfaction, dissatisfaction, and confirmation
May hesitate and even have to change the subject because of lack of language resources
Simple structures and basic grammatical relations are only somewhat controlled
Time references may be used incorrectly
Vocabulary may be imprecise except for the highest frequency utterances
Frequent errors in pronunciation, vocabulary, or grammar may impede communication
Delivery may be labored
Strings of related sentences

12. 2
FUNCTIONAL
Past narration
Description
Report on a current event
Instructions /directions
Deal with a familiar situation with a complication RP
IGT in Past tense
Can narrate current, past and future activities
Can describe people, places, and things
Can create full paragraphs, minimally cohesive
Can confidently handle most normal, casual conversations on concrete topics
Can often elaborate in common daily communicative situations
Can give complicated and detailed directions and make non-routine changes in travel and other arrangements
Simple structures and basic grammatical relations are typically controlled, while more complex structures are used inaccurately or avoided
Vocabulary use is appropriate for high-frequency utterances but unusual and imprecise at other times
Errors in pronunciation, vocabulary, and grammar may sometimes distort meaning
Generally speaks in a way that is appropriate to the situation, although command of the spoken language is not always firm
Full paragraphs, minimally cohesive 2+
FUNCTIONAL+
Three probes at level 3
Can create a significant amount of discourse beyond the paragraph level
Can use the language effectively to narrate, describe, state facts, compare and contrast, give detailed instructions and directions
Can use the language with less ease and effectiveness to support opinion, clarify points, and answer objections
Shows some linguistic limitations when dealing with unfamiliar subjects and situations
Can generally elicit information and informed opinion
Can often discuss abstract concepts but can rarely use abstract linguistic formulations successfully
Errors in vocabulary and more complex structures and cohesive features sometimes interfere with efforts to elaborate on an argument or point of view
Pronunciation errors may occasionally impede communication
Speech is usually appropriate to the situation
Discourse beyond the paragraph level 3
PROFESSIONAL
Supported opinions
Hypothesizing
Discuss an abstract topic
State and defend Policy
Clarify points
Produces extended discourse and conveys meaning correctly and effectively
Use of structural devices is flexible and elaborate
Speaks readily and in a way that is appropriate to the situation
Without searching for words or phrases, can use the language clearly and relatively naturally to elaborate on concepts freely
Can convey abstract concepts
Can easily repair the conversation
Pronunciation may be foreign
Errors may occur in low frequency or highly complex structures characteristic of a formal style of speech
Extended discourse

13. Fluency & Accuracy Two components of assessment
Opposite ends of a continuum in which at extremes speech is seen as accurate and disfluent or inaccurate and fluent
Two components of assessment
Accuracy of structure and vocabulary
Quality and speed of delivery

14. ‘Error gravity’ – determines seriousness of errors
- John buy a newspaper every day.
- Every day a newspaper buy.
(James,1998)

15. Definitions of fluency
Fluency markers
flow of smoothness
rate of speech
absence of excessive pausing
absence of disturbing hesitation markers
length of utterances
connectedness

16. Central part of fluency
Speaking rate
Speech –pause relationships
Frequency of dysfluency markers
( hesitations, repetitions, self-corrections)
“Small words” and generic words

17. High frequency concrete Voc- eyeglasses; mast; etc. Simple structures
Range of Vocabulary
Grammar Structures
Level 2
High frequency concrete Voc- eyeglasses; mast; etc.
Simple structures
Level 3
Low frequency concrete voc - spectacles
High frequency abstract Voc -obligation; reduction; permanent; dilemma
Complex grammatical structures
Level 4
Low frequency concrete voc – monocle; mizzenmast ; camshaft
Low frequency abstract- conundrum; scoff; abhor; rebuke.
Low frequency grammatical structures

18. Bibliography Testing Second Language by Glenn Fulcher
Assessing Speaking by Sari Luoma
Common European Framework of Reference for Languages: Learning, teaching, assessment
Standardization Agreement 6001 (5th addition )
LTS/ALTS material (Trisection of speaking and OPI checklist)

19. Thank you