The Place of Direct and Indirect Tests in the Assessment of Speaking Skills: A Case Study Dr. Hale Kızılcık May 30, 2012.

Slides:



Advertisements
Similar presentations
Performance Tasks for English Language Arts
Advertisements

An ESL Learner in Transition University of Sydney 4 November 2013.
Testing What You Teach: Eliminating the “Will this be on the final
 The award of satisfactory completion for a unit is based on a decision that the student has demonstrated achievement of the set of outcomes specified.
Writing Tasks on the Literacy Test  On the literacy test, you will have two long writing tasks  1) A News Report  and  2) A Series of Paragraphs (Opinion.
Distance Learning Foundation and US Embassy RELO Office Session #3 Unlocking Learners’ Voices 3 February 2011.
Writing Performance Tasks* for CTE *also known as Writing Tasks.
TYPES OF TEST ITEMS/TASKS
IB Diploma Program Exams – Semester Report Cards
Assessment and Rubrics From K. Purgason and J. Adelson-Goldstein.
+ Classroom Expectations & Procedures Mr. Chavez Speech Fall 2014-F205.
EFFECTIVE GROUP PRESENTATIONS creating unity from diversity.
The Importance of Learning Strategies in ELT Sandra J. Briggs TESOL Past President
Speaking Objectives  Talk about routines & campus life  Express ideas and opinion  Giving presentation  Participate in seminars & discussions  Delivery.
Technical Report Writing and Presentation Skills Course Outline 1.
Lecture 8 Assessing Listening Chapter Six Pages: Brown, 2004.
RUAMRUDEE INTERNATIONAL SCHOOL
Module 1 Unit 2 Project: writing an advice letter --By Zhou Zhenghu No
Discussions and Oral Presentations as Teaching Material in English for Medicine Zorica Antic Natasa Milosavljevic English language department Faculty of.
Integrated Approach to Project Work: Effective Use of “ Relay Method ” and Technology Yuka Akiyama CAS Japanese instructor SED graduate student in TESOL.
Assessment (Midterm /Finals) In-class Participation15 %Homework15%Quizzes20% Project10%Test40% Grade 3.
Sarah Peterson Amy von Barnes Making “I Can” Statements Easy Supporting Learners – Week 2.
FROM LANGUAGE NEEDS ANALYSES TO CLASSROOM INSTRUCTION Nana Toradze Language Training School National Defense Academy, Georgia
Assessment CLEAR & UNAMBIGUOUS. What is the purpose of your assessment? *************************************** To evaluate of overall proficiency? For.
UNDERSTANDING THE OUTCOMES POSTER ACTIVITY. THE STEPS FOR THE ACTIVITY: You will be placed in groups of 4. Each group will start with an outcome. The.
Asking the Right Questions Assessing Language Skills 2008 Presentation to ATESL Central Local Sheri Rhodes, Mount Royal College.
1 Performance Assessment in American Foreign Language Classrooms Dr. Paula Winke FLTA Orientation August 10, 2011 Parlor C, MSU Union.
+ What is th CELDT? What you need to know to be successful on this important exam.
Developing essential skills for academic success Mike S. Boyle mikesboyle.com
ACADEMIC SKILLS IN ENGLISH-1 ENG 101 ECONOMICS AND ADMINISTRATIVE SCIENCES
The testing process Decision-making in test creation.
Fall 2005 English Communication (ENG101 X 3) This course introduced students to everyday English and provided them with a strong foundation in the language.
Fitch High School February 6, 2014 Fitch High School February 6, 2014.
Lecture 7. The Questions: What is the role of alternative assessment in language learning? What are the Reasons.
S&T Student Course Evaluations: Beyond Question 7 – Part 2 Dan Cernusca, Ph.D. Instructional Design Specialist Missouri University of Science and Technology.
Opposing Viewpoints Teaching American History In Miami-Dade County December 14, 2012 Fran Macko, Ph.D.
“I Can” Learning Targets 4 th English/Writing 5th Six Weeks.
Performance Task. Thursday What is it? A test Gauges your reading, discussion and writing skills Divided into 3 parts 1. Reading, understanding and annotating.
Understand the purpose and benefits of guiding instructional design through the review of student work. Practice a protocol for.
Constructed Response Developing this writing practice as part of ongoing classroom assessment The value of constructed response is that it is teaching.
Ms.Samina Javed The City School Mathematics,Grade 5
IELTS Intensive Writing part two. IELTS Writing Two parts of ielts writing Part one writing about a Graph, chart, diagram Part two is an essay.
Week 7 Debates. On a separate sheet of paper, Who is on your debate team? (Not the topic, but your specific stance) In your opinion, who are the best.
Greenbush. An informed citizen possesses the knowledge needed to understand contemporary political, economic, and social issues. A thoughtful citizen.
Taking the Mystery out of the Bilingual Target Language Proficiency Test (BTLPT): Why Candidates Are Failing the Test? Maria Bhattacharjee Ed.D Irene Chen.
What to Expect When Expecting ESL Students: Practical Suggestions for Accommodating English Language Learners in the Regular Classroom Created by Jenny.
STATE OF TEXAS ASSESSMENTS OF ACADEMIC READINESS (STAAR TM ) Grades 3−8 Reading Grades 4 and 7 Writing English I, II, and III Victoria Young Director of.
“I Can” Learning Targets 4 th English/Writing 6th Six Weeks.
Candidate Support. Working Agreements Attend cohort meetings you have agreed upon. Start and end on time; come on time and stay for the whole time. Contribute.
Unit 5 Wealth and Happiness Reading 1 Three Old Men and a Woman.
Writing is Essential: Overview for Student Success Presented by Angela McClary-Rush WCSD, ELA Coordinator.
Activities to Promote Speaking. Speaking is "the process of building and sharing meaning through the use of verbal and non-verbal symbols, in a variety.
Final Exam Specifications. Your final exam will consist of three sections: A) Reflection, B) An achievement test and C) Appendix. You must submit your.
Assessing Speaking. Possible challenges in assessing speaking Effect of listening skill: Speaking without interaction is observable but very limited (telling.
Common Core.  Find your group assignment.  As a group, read over the descriptors for mastery of this standard. (The writing standards apply to more.
English III. Learning Objectives The course involves a careful instruction of English as both an academic subject and a foreign/second language, and is.
International & Comparative Education: Education in Majority-English-Speaking Countries Administration.
Dr Anie Attan 26 April 2017 Language Academy UTMJB
Learning the Nuances of Written English by fusing the Strategies of Spoken English Learning: An Approach to EFL Students.
International & Comparative Education: Education in Majority-English-Speaking Countries Administration.
ASSESSING PRODUCTIVE SKILLS AT B2. UPDATED FIRST WRITING AND SPEAKING
Integrating Layered Delivery within the Learning by Doing Model
A course in English for Academic Purposes Intermediate/B1+
Listening.
Back to School Night Mme Susan Boyle
Lesson 21: Timed writing About this lesson
“I Can” Learning Targets
Improving academic performance Building language skills Developing critical thinking Expressing ideas and opinions Ask the audience: What are the core.
FCE IES Parque de Lisboa.
Presentation transcript:

The Place of Direct and Indirect Tests in the Assessment of Speaking Skills: A Case Study Dr. Hale Kızılcık May 30, 2012

OUTLINE ENG 211 Academic Speaking Skills The development of the midterm exam A closer look at the exam: «Discussion skills» Exam results Interpreting the exam results

ENG 211 Academic Speaking Skills Revised in Not only to focus on «speaking as performance» (presentation) But also to focus on «speaking as transaction» (Richards, 2007)

Assessment PRESENTATIONS 50% Team Presentation Informative Presentation Final Presentation 10% 30% SPEAKING ACTIVITIES20% Current Events News Story Going One Step Further Tasks 5% 15% MIDTERM EXAM20% WRITING ASSIGNMENTS 10% Speech Analysis Report Self-evaluation Report 5%

Preparing a written test for a speaking course Valid Reliable Authentic Practical Backwash (Hughes, 1998)

Course objectives Cooperation with the course coordinators Face validity

Parts of the Midterm Exam Listening for specific information and main ideas (podcast) 6 pts. Synthesizing & expressing opinions (6 pts.) *Listening for note-taking Organizing a speech (4 pts.) Discussion skills (4 pts.)

Discussing testing «discussion skills»  «They already do it in the class (a lot of times!)»  «Is it a good idea to test spoken language in written discourse?» «It may have benefical backwash effect. It can encourage students to pay attention to signposts» «They already do it... But let’s see if they can integrate sources and use appropriate strategies» «In this way, we can also test their critical thinking skills»

Part IV: Discussion Skills (4 pts.) The following excerpts are taken from discussions. Participate in each discussion as instructed. Express your views in 3-4 sentences. (2 pts. each) A: Read the transcript of a discussion on graffiti. Interrupt the last speaker appropriately, agree with Victoria, and support your stance by giving your own reasons. Victoria: I think one should consider graffiti from a broader angle. Graffiti is not limited to painted large letters on walls. Think of a graffiti artist like Banksy. I would love to have one of his paintings on my wall. Sadi: I do not think graffiti can be justified on any grounds. It is a form of vandalism and personally I would sue anybody, Banksy or Picasso, who would paint graffiti on the walls of my house. You: ______________________________________________________________

Sample answer: I am sorry but I’d like to make a point here. I think Victoria is right and we have to distinguish between different forms of graffiti. Besides, I’d also really appreciate a Banksy on my wall. First, his graffiti is full of important political and social messages which are worth spreading. Second, they cost a fortune. One of his graffiti works can make me a rich man. Criteria for marking: Signpost and stance: 0.5 pt. Support: 1.5 pts.

Testing Discussion Skills Directly Sample speaking task & Debate & discussion rubric

Criteria Quality of arguments (5 pts.) Style (5 pts.) Delivery (5 pts) Language (3 pts.) Team work (2 pts.)

Interpreting the Midterm Results

Success Rate in Different Sections of the Exam Part I (listening): 61% Part II-A (synthesizing): 72% Part II-B: (expressing opinions): 77% Part III: (organizing a speech): 78% Part IV-A: (discussion skills): 49.5% Part IV-B: (discussion skills): 49.5%

CHART DISPLAYING ST. GRADES IN PART IV AND IN-CLASS DISCUSSIONS

So? What does this mean? Is there a problem with the reliability of in-class speaking tests? Is there grade inflation on the part of the teacher? Is there a problem with the discussion grading rubric? Is there a problem with the validity of the written test? Are the two tests testing different skills?

A closer look at the extreme cases Grammar becomes more important in the indirect test. Repetitions & digressions may be overlooked in the direct test. Quality of ideas are scrutinized more in the indirect test. The criteria for the direct test is broader in scope. There are personality issues.

Conclusions On testing speaking indirectly On testing speaking as performance On structures speaking tasks & automatic scoring of tests (Luoma, 2004) On teaching discussion skills

Thank you

REFERENCES Hughes, A. (1989). Testing for language teachers. Cambridge: Cambridge University Press. Luoma, S. (2004). Asessing speaking. Cambridge: Cambridge University Press. Richards, J., C. (2007). Developing Classroom Speaking Activities: From Theory to Practice: Summary article. From classroom-speaking-activities.pdf