The testing process Decision-making in test creation

Golden Rule No testing without thinking! Decide what it is that you want to know (measure) and keep that in mind throughout the test development process

Three Key Concepts Validity Reliability Practicality

Define the domain What am I trying to test? –Speaking ability What aspect of speaking ability? –Discrete skills versus integrative? –General sample of language? –Performance on specific task? Order a meal at a restaurant Give a presentation in class

Consider possible tasks Goal: to assess general speaking ability Five-minute interview –Spontaneous versus guided –Use of prompts Giving directions Picture description Picture sequence (“What happened?”, make a story) Topical discourse (recent events, vegetarianism) Situations (Here is a menu. Order a pizza.)

Consider evaluation criteria Pass / Fail Impressionistic scoring Holistic scoring based on rubric –Good = accent not distracting, uses appropriate language and vocabulary, fairly fluent delivery Analytic scoring –Fluency = 10 pts. –Accuracy = 10 pts. –Appropriate vocabulary = 10 pts. –Appropriate politeness = 10 pts. –Willingness to communicate = 10 pts.

Administration procedures When and where? –During class in the back of the room? –After school in the library? How? –Warm-up / cool-down period? –Tape or video recorded? –Checklist for grading?

Reporting procedures What will the student receive? –A grade –a comment about their performance –Extra homework

Criterion

The authenticity question There is always a gap between the test and the criterion Direct versus indirect versus real