XIV Peru-TESOL Sharing Teaching and Learning Experiences in a Global World Iquitos August 2, 2006 M. Karen Jogan, Ph.D.

Catching that elusive butterfly: assessing speaking in five easy steps

M. Karen Jogan, Ph.D. Peru TESOL Today we will… Outline the process to capture oral competency of EFL students Define oral proficiency – identify proficiency levels in instructional program Design assessable speaking tasks for each level Identify or create rubrics used to assess speaking tasks Align assessment tool to a program outcome Prepare to implement an oral assessment plan

M. Karen Jogan, Ph.D. Peru TESOL What are the steps in the process? 1.Determine minimal outcomes, using proficiency guidelines 2.Include speaking tasks for assessment in the classroom 3.Adapt or develop assessment tools to assess speaking 4.Promote contextualized speaking activities 5.Assess the speaking task, incorporate into overall assesment framework

M. Karen Jogan, Ph.D. Peru TESOL Determine minimal student outcomes

M. Karen Jogan, Ph.D. Peru TESOL What is proficiency? Novice Intermediate Advanced Superior

M. Karen Jogan, Ph.D. Peru TESOL Specific linguistic functions

M. Karen Jogan, Ph.D. Peru TESOL Superior Advanced Intermediate NoviceTell time, give one´s name, describe weather, list objects

M. Karen Jogan, Ph.D. Peru TESOL Superior Advanced Intermediate Ask for directions, describe preferences, ask and answer questions, order a meal NoviceTell time, give one´s name, describe weather, list objects

M. Karen Jogan, Ph.D. Peru TESOL Superior Advanced Narrate a story in the past, compare and contrast Intermediate Ask for directions, describe preferences, ask and answer questions, order a meal NoviceTell time, give one´s name, describe weather, list objects

M. Karen Jogan, Ph.D. Peru TESOL SuperiorSupport opinion, hypothesize, narrate in detail, abtract topics Advanced Narrate a story in the past, compare and contrast Intermediate Ask for directions, describe preferences, ask and answer questions, order a meal NoviceTell time, give one´s name, describe weather, list objects

M. Karen Jogan, Ph.D. Peru TESOL Discourse – from limited to extended

M. Karen Jogan, Ph.D. Peru TESOL Superior Discourse level Multiple paragraphs Advanced Multiple sentences paragraph level Handle complex situation and transaction Sentences create with language; handle simple situation/transaction Novice Short memorized phrases lists

M. Karen Jogan, Ph.D. Peru TESOL Time to think! In which level of EFL instruction are your students able to perform the following?  Complain  Summarize  Narrate a past event  Greet a friend  Compare  Ask questions to elicit personal information

M. Karen Jogan, Ph.D. Peru TESOL To consider how well students perform language functions, which (assessment) activities are appropriate?  What does the student DO to demonstrate competency???

M. Karen Jogan, Ph.D. Peru TESOL Describe Ask and answer questions Narrate in past

M. Karen Jogan, Ph.D. Peru TESOL Describe Ask and answer questions Narrate in past Oral presentations Interview your partner Story retelling, summarizing Picture-cued descriptions

M. Karen Jogan, Ph.D. Peru TESOL Ask/provide information Agree/disagree Express opinions Support opinions Persuade Complain

M. Karen Jogan, Ph.D. Peru TESOL Ask/provide information Agree/disagree Express opinions Support opinions Persuade Complain Role Plays Debates Information gap

M. Karen Jogan, Ph.D. Peru TESOL Get the gist Get the main idea Get specific details Summarize

M. Karen Jogan, Ph.D. Peru TESOL Get the gist Get the main idea Get specific details Summarize Tell/discuss what you heard  audio file  film  news report  native speaker… Tell about what you read in  newspaper report  story, etc.

M. Karen Jogan, Ph.D. Peru TESOL Summarize Describe Synthesize Analyze Summarize/describe/ anaylze  video  program  Article, etc.

M. Karen Jogan, Ph.D. Peru TESOL Include speaking tasks for assessment in the classroom

M. Karen Jogan, Ph.D. Peru TESOL List all speaking activities which are part of your course Designate or design specific speaking tasks for assessment Define all elements of the “product” which you expect your students to produce Consider which criteria you will use for assessment

M. Karen Jogan, Ph.D. Peru TESOL Interpersonal mode  Roleplays among students  Interviews between students  Oral monologues  Situation cards (simulations)  Performing a play Presentational mode  Narration or description (oral presentation)  Poster session  Oral presentation with power point

M. Karen Jogan, Ph.D. Peru TESOL Time to talk! Describe an oral activity which your students have performed successfully. Why do you think that activity was successful? Describe an oral activity that your students did not perform well. Why do you think that the activity was unsuccessful?

M. Karen Jogan, Ph.D. Peru TESOL Rules to remember Design the task carefully. If you can´t define it, you can´t assess it.

M. Karen Jogan, Ph.D. Peru TESOL Designing a speaking activity Determine level Select function or language task  What is the grammar students need to perform this task? Select a topic or situation  Vocabulary? Define a time frame (present/past/future) Define the elements of the context  Who, what, where, when??? What aspects of culture must be included as part of the activity?

M. Karen Jogan, Ph.D. Peru TESOL It´s your turn! Design a speaking activity 1.Level 2.Function 3.Grammar 4.Context – who, what, where, when 5.Content – topic or situation 6.Culture

M. Karen Jogan, Ph.D. Peru TESOL Adapt or develop assessment tools to assess speaking

M. Karen Jogan, Ph.D. Peru TESOL Recognize that a variety of assessment tools can be used Survey avaliable assessment instruments Examine criteria; recognize that criteria can differ according to instructional level Collaborate to develop appropriate assessment tools Match assessment tool to appropriate activity and instructional level

M. Karen Jogan, Ph.D. Peru TESOL Promote contextualized speaking activities

M. Karen Jogan, Ph.D. Peru TESOL Move from skill building to skill using in the instructional program  What can students DO???? Move from meaningful to communicative to situational activities Provide a framework for students to structure a conversational exchange Monitor the process

M. Karen Jogan, Ph.D. Peru TESOL Sample role play buildup Context:  You have just met a new student from the U.S. who is attending your university. You want to find out about this person and what you both have in common. After you establish common ground, invite your new friend to join you for an activity in which you are both interested.

M. Karen Jogan, Ph.D. Peru TESOL Prepare your students for success!

M. Karen Jogan, Ph.D. Peru TESOL Sample role play buildup: instructions to students Preliminary step  Key words and phrases I will use  Question words  Nouns  Verbs

M. Karen Jogan, Ph.D. Peru TESOL Sample role play buildup: instructions to students 1.Write down the question words you will need to ask 2.Write down possible answers to these questions 3.Interview someone in the class 4.Write down the answers you hear 5.Let your partner interview you

M. Karen Jogan, Ph.D. Peru TESOL Sample role play buildup: instructions to students 6.Decide with your partner which questions are the most appropriate 7.Decide with your partner which role each of you will play 8.Practice acting out with your partner

M. Karen Jogan, Ph.D. Peru TESOL And the final step Now, act out your role play.  Teacher uses assessment tool to evaluate the student product

M. Karen Jogan, Ph.D. Peru TESOL Excedes requirements Meets requirements Needs improvement Content Original and on task: excellent use of new vocabulary Content is complete Incomplete content or vocabulary Accuracy Grammar usage is exceptional Grammar usage is appropriate Grammar needs improvement Fluency Speaks without hesitations Some hesitations Halting and many hesitations

M. Karen Jogan, Ph.D. Peru TESOL Assess the speaking task and incorporate into the overall assessment framework

M. Karen Jogan, Ph.D. Peru TESOL Address the goals of your program Make known to students the criteria for assessment and the specific tasks to be assessed prior to the activity Apply an appropriate assessment instrument to the oral task Review outcomes

M. Karen Jogan, Ph.D. Peru TESOL And finally… Evaluate the effectiveness of your assessment tool Consider how to improve your course or program, based on results from assesment

M. Karen Jogan, Ph.D. Peru TESOL Programatic global assessment Oral interview Oral presentation – final project Essay – final product Listen or read and summarize/evaluate/analyze orally or in writing Portfolio = collections, selections, and reflections Standardized test or mock test Integrative tests tap all skills!

M. Karen Jogan, Ph.D. Peru TESOL How can we catch that elusive butterfly? 1.Build a team 2.Determine goals of your assessment program. 3.Match course objective to the test. 4.Test like you teach, teach like you test! 5.Review sample assessment tools. 6.Adapt tools to meet your program’s needs

M. Karen Jogan, Ph.D. Peru TESOL Use the assessment tool with your class. 8.Critique the tool with your colleagues. Are you consistent with your colleagues in its application? Are there ways that the tool or the activity could be improved? Should more “buildup” opportunities be provided to your students? 9.Collaborate and continue to share, review, adapt, & critique with colleagues.

M. Karen Jogan, Ph.D. Peru TESOL

M. Karen Jogan, Ph.D. Peru TESOL Time to work…

M. Karen Jogan, Ph.D. Peru TESOL Your Assignment… Define a student outcome for oral production Clearly define the task that the student will complete to demonstrate competency Identify a rubric for use in evaluating the task. Consult online resources that follow. Match rubric to task

M. Karen Jogan, Ph.D. Peru TESOL Online references Rubistar  Adapt rubrics to meet the criteria you determine to evaluate student products Rubric Bank – Chicago City Schools  Review grading guidelines for speaking, writing activities cs/ideas_and_rubrics.html cs/ideas_and_rubrics.html Landmark  Create a rubric from scratch project.com/classweb/tools/rubric_builder.php3 project.com/classweb/tools/rubric_builder.php3

M. Karen Jogan, Ph.D. Peru TESOL University of Alaska sample assessment instruments  Scoring guides linstr3.html linstr3.html Kenyon County Schools  Assesment tools and links

M. Karen Jogan, Ph.D. Peru TESOL Discovery School Assessment tools  Comprehensive collection of assessment tools l l Fairfax County Virginia Public Schools  Performance Assessment for Language Students /rubrics/ /rubrics/

M. Karen Jogan, Ph.D. Peru TESOL References Bachman, Lyle F Fundamental considerations in language testing. New YOrk: Oxford University Press Oller, John W Language tests at school. New York: Longman O’Malley, J. M. and L. Valdez Pierce (1996). Authentic Assessment in English Language Learners. New York; Addison Wesley.