PRESENTER: TRAN THI HIEU THUY SUPERVISOR: DR. TO THI THU HUONG EVALUATION OF AN END-TERM LISTENING TEST FOR FIRST YEAR MAINSTREAM STUDENTS OF ENGLISH DEPARTMENT – COLLEGE OF FOREIGN LANGUAGES – VNU PRESENTER: TRAN THI HIEU THUY SUPERVISOR: DR. TO THI THU HUONG

DESIGN OF THE STUDY Introduction Development Conclusion Chapter I: Literature Review Chapter II: The Study Chapter III: Data Analysis, Discussion of findings and Recommendations Conclusion

Rationale Compilation and implementation of “Practise your listening skills” for first-year students of ED: easy-to-follow organization, stimulating contents and useful inputs Change in assessment: Listening portfolio (35%) + Mid-term test (15%) + End-term test (50%) Realization of the importance of incorporating authentic data into teaching listening (Nunan, 1999) Little concern paid to the washback effects of the end-term test on teachers and students of Division 1 - ED Evidences from worldwide studies suggesting the existence of test impacts on teachers and students (Shohamy et.al. (1996), Cheng (1995) and Wantanabe (1996) ) Topic for MA thesis: Evaluation of an end-term listening test for first year mainstream students of English Department – College of Foreign Languages – Vietnam National University

Scope of the study Washback effects of the end-term listening test of the second semester that K41 students (academic year 2007-2008) sat for on teachers and students.

Research questions (1) What are the washback effects of the test on teachers’ teaching content, teaching methodology, attitudes and behaviors? Are they positive or negative? (2) What are the washback effects of the test-taking experience and test results on K41 students in terms of their learning content, learning progress, self-image, motivation, learning attitudes and their relationship with teachers? Are they positive or negative?

Chapter I: Literature Review I.1. Language testing in communicative approach I.2. Achievement Language Test I.3. Testing listening comprehension skill

Washback effects of a test (Hughes, 2001, p.1): The effect of testing on teaching and learning is normally referred to as backwash Synonymous term: washback (Buck (1988, p.17), Shohamy (1992, p.513), Bachman and Palmer (1996, pp.29-35), and so on). Messick (1996, p.241): For a test to have positive washback, its tasks should be criterion samples – that is, “authentic and direct samples of the communicative behaviors of listening, speaking, reading and writing of the language being learnt”

Washback effects of a test Bailey (1999): Washback process may concern: Teachers Students Test developers Teacher educators Curriculum planners Administrators Language inspectors Test users Material developers and publishers Parents

Chapter II: The Study II.1. BACKGROUND TO THE STUDY: THE CONTEXT OF TEACHING AND TESTING ENGLISH LISTENING COMPREHENSION OF FIRST YEAR STUDENTS II.1.1. Standard levels of mastery of Listening comprehension skill for first year students II.1.2. An overview of the second-semester English listening program and its testing in Division 1 – English Department – CFL – VNUH II.1.2.1.The second-semester listening program in Division 1 - English Department – CFL II.1.2.2. The testing of listening comprehension for first year students in English Department - CFL

Chapter II: The Study II.2. METHODOLOGY OF THE STUDY II.2.1. A qualitative and quantitative research II.2.2. Subjects II.2.3. Methods and procedures of data collection II.2.4. Methods of data analysis

Chapter II: The Study II.2. METHODOLOGY OF THE STUDY II.2.1. A qualitative and quantitative research II.2.2. Subjects II.2.3. Methods and procedures of data collection II.2.4. Methods of data analysis 50 K41 students 12 teachers of Division 1 – English Department

Data for the study were collected from 6 sources: Chapter II: The Study II.2. METHODOLOGY OF THE STUDY II.2.1. A qualitative and quantitative research II.2.2. Subjects II.2.3. Methods and procedures of data collection II.2.4. Methods of data analysis Data for the study were collected from 6 sources: Questionnaire survey for K41 teachers; Questionnaire survey for K41 students; Class observations of two classes; Interviews with two randomly selected teachers; Interviews with two randomly selected students; Scores from the second semester end-term listening test.

Chapter II: The Study II.2. METHODOLOGY OF THE STUDY II.2.1. A qualitative and quantitative research II.2.2. Subjects II.2.3. Methods and procedures of data collection II.2.4. Methods of data analysis Two groups of data: teachers’ responses and students responses Themes ascertained Qualitative analysis: Text rendering (D’Arbon, 2008) Quantitative analysis: Frequency, percentage, charts

Chapter III: Data Analysis, Discussion of findings and Recommendations RESEARCH QUESTIONS (1) What are the washback effects of the test on teachers’ teaching content, methodology, attitude and behavior? Are they positive or negative? (2) What are the washback effects of the test-taking experience and test results on K41 students on their learning content, learning progress, self-image, motivation, learning attitudes and their relationship with teachers? Are they positive or negative?

Teaching content Teachers’ responses Fulfillment of class teaching to the listening syllabus Types of activities carried out during class time Teaching materials

Teaching content Teachers’ responses Total commitment of listening teachers to the syllabus Homework checking and In-class practice: essential parts of the lesson Taught skills: as suggested in the syllabus Test covering learnt knowledge and skills Use of past test papers and IELTS practice tasks as teaching materials Test mentioning and practicing: increase as the test came nearer

Teaching methodology Teachers’ responses Following stages of a lesson and procedure of carrying out the main tasks Maintaining sufficient time for classroom activities Being willing to adjust teaching methodology Mentioning the end-term test to draw students’ attention to the lesson Spending more time for test practice and feedback in the last weeks of the semester

Attitudes and behaviors Teachers’ responses Teachers’ anticipation of changes on their part when having a detailed report of students’ past test scores

Impacts of test-taking experience Students’ responses Students’ perception of task 2 in the test Students’ decision on changing learning contents

Task 2 in the test Students’ responses

Changes in learning contents Students’ responses

The test result influences … Impact of test result Students’ responses The test result influences … LSG ASG HSG Your learning attitudes 6 2 4 Your learning strategies 3 Your motivation for learning this skill 5 Your self-image Your learning progress in this skill 1 Your relationship with teacher

Findings – Answer to question 1 Washback effects of the test on teachers: Not “teaching-to-the-test” A bit much of teachers’ talk about the test  More evidences of positive washback effects than negative ones

Findings – Answer to question 2 Washback effects of the test on students: complicated Development of learning strategies Preparation of study plan Desire to earn good mark Evidence of “learning for the test”

Recommendations to teachers Guiding students to effective use of previous test papers Encouraging students to seek for fun in learning listening Being more sensitive with the content of teacher talk in class Using teaching materials wisely Constructing item-analysis report

Recommendations to students Taking it easy when dealing with the test Using learning materials effectively Developing the habit of analyzing weak points and making plans to improve them Seeking for help and support from teachers and peers

Conclusion Comprehensive responses to research questions Significant contribution to the deep evaluation of one criterion of a good test

THANK YOU FOR YOUR ATTENTION!!!

INTRODUCTION Rationale Aims and objectives Scope of the study Research questions Research methodology Significance of the study

Aims and objectives (1) Investigating the washback effects that the second end-term listening test has on teachers of listening skill for K41 students; (2) Investigating the washback effects that the second end-term listening test has on K41 students of English department; (3) Evaluating whether such evidences of washback (if any) are positive or negatives to teachers and students; (4) Proposing ways to enhance the positive impacts and to minimize the negative effects of the test on teachers’ teaching and students’ learning.

Research methodology 50 students 12 teachers Quantitative and qualitative approach Data collection tools: - Semi-structured interviews - Questionnaires - Classroom observations Subjects: 50 students 12 teachers

Significance of the study provides a reliable and profound background on the matter helps the researcher gain more knowledge and skills in this field offers an insight into test washback, an area rarely investigated in a university educational context highlights the close relationship between teaching - learning and testing

Chapter I: Literature Review I.1. Language testing in communicative approach I.1.1. What is a language test? I.1.2. Testing in Communicative Approach I.1.3. Purposes of language testing I.1.4. Washback effect of a test I.1.4.1. Washback: definitions and types I.1.4.2. Test-takers and washback I.1.4.3. Teachers and washback I.1.4.4. Other participants and washback

Language testing Allen (1974, p.313): “a test is a measuring device which we use when we want to compare an individual with other individuals who belong to the same group.” Heaton (1988, p.5): tests are considered first as “means of assessing the students’ performance” and then as devices to “motivate students.”

Communicative competence Canale (1983, p.339, cited in Tran, 2002, p.5) proposes four dimensions of CC: - Grammatical competence - Sociolinguistic competence - Discourse competence - Strategic competence

Purposes of language testing For teachers evaluate reveal teaching effectiveness provide additional materials For students detect strong points and weak points be a motivation

Chapter I: Literature Review I.2. Achievement Language Test I.2.1. Definition I.2.2. Kinds of achievement tests

Chapter I: Literature Review I.2. Achievement Language Test I.2.1. Definition I.2.2. Kinds of achievement tests Hughes (2001, p.10): “…achievement tests are directly related to language courses, their purpose being to establish how successful individual students, groups of students, or the course themselves have been in achieving objectives.”

Chapter I: Literature Review I.2. Achievement Language Test I.2.1. Definition I.2.2. Kinds of achievement tests Two kinds of achievement tests: - Progress achievement tests - Final achievement tests

Chapter I: Literature Review I.3. Testing listening comprehension skill I.3.1. The nature and sub-skills of listening comprehension I.3.2. Testing listening comprehension

Chapter I: Literature Review I.3. Testing listening comprehension skill I.3.1. The nature and sub-skills of listening comprehension I.3.2. Testing listening comprehension Traditional approach Alternative approach McDonough & Shaw (1993): Processing sounds Processing meaning

Chapter I: Literature Review I.3. Testing listening comprehension skill I.3.1. The nature and sub-skills of listening comprehension I.3.2. Testing listening comprehension Kitao and Kitao (1996, p.1): Understanding sentences and dialogues: Interpreting meaning, responding to utterances Task using visual materials: Matching, T/F, Mapping, Drawing Tasks involving talks and lectures: Summary-filling, Matching, T/F, Short answers, MCQs, Note-taking, etc.

Motivation to learn listening skill Students’ responses Factor Yes No 1 Necessary for future job 48 100% 2 Interest in English 40 83.3% 08 17.7% 3 Compulsory 18 37.5% 30 62.5%

Teaching methodology Teachers’ responses Frequency of teachers following teaching process and conducting the main task in a procedure

Teaching methodology Teachers’ responses Teachers’ reference to the end-term test to draw students’ attention

Students’ responses * low-scored group (LSG): five students (<5 points) * average-scored group (ASG): 24 students (5 to 6 points) * high-scored group (HSG): 19 students (7 to 10 points:18 students of 7-8 points and 1 student with 9-10 points)

Frequency of activities in listening lessons Students’ responses

Teaching methodology Process of lesson Reference to the end-term test Teachers’ responses Process of lesson Reference to the end-term test

Students’ past test score Teachers’ responses Teachers’ care about students’ past test score Teachers’ preference of having item-analysis report of students’ past test scores

Students’ in-class performance Teachers’ responses Interest in talks about test-taking techniques and test practice High appreciation of test practice in-class High appreciation of the importance of the end-term test

Classroom observation Time: Week 13, 14, 15 of the second semester Comments: Maintenance of the process of a listening lesson Fulfillment of requirements of the course outline. Times of mentioning the end-term exam and discussing test-taking techniques No exam pressure put on students.