1. PRESENTER: TRAN THI HIEU THUY SUPERVISOR: DR. TO THI THU HUONG
EVALUATION OF AN END-TERM LISTENING TEST FOR FIRST YEAR MAINSTREAM STUDENTS OF ENGLISH DEPARTMENT – COLLEGE OF FOREIGN LANGUAGES – VNU
PRESENTER: TRAN THI HIEU THUY
SUPERVISOR: DR. TO THI THU HUONG

2. DESIGN OF THE STUDY Introduction Development Conclusion
Chapter I: Literature Review
Chapter II: The Study
Chapter III: Data Analysis, Discussion of findings and Recommendations
Conclusion

3. Rationale
Compilation and implementation of “Practise your listening skills” for first-year students of ED: easy-to-follow organization, stimulating contents and useful inputs
Change in assessment: Listening portfolio (35%) + Mid-term test (15%) + End-term test (50%)
Realization of the importance of incorporating authentic data into teaching listening (Nunan, 1999)
Little concern paid to the washback effects of the end-term test on teachers and students of Division 1 - ED
Evidences from worldwide studies suggesting the existence of test impacts on teachers and students (Shohamy et.al. (1996), Cheng (1995) and Wantanabe (1996) )
Topic for MA thesis: Evaluation of an end-term listening test for first year mainstream students of English Department – College of Foreign Languages – Vietnam National University

4. Scope of the study
Washback effects of the end-term listening test of the second semester that K41 students (academic year ) sat for on teachers and students.

5. Research questions
(1) What are the washback effects of the test on teachers’ teaching content, teaching methodology, attitudes and behaviors? Are they positive or negative?
(2) What are the washback effects of the test-taking experience and test results on K41 students in terms of their learning content, learning progress, self-image, motivation, learning attitudes and their relationship with teachers? Are they positive or negative?

6. Chapter I: Literature Review
I.1. Language testing in communicative approach
I.2. Achievement Language Test
I.3. Testing listening comprehension skill

7. Washback effects of a test
(Hughes, 2001, p.1): The effect of testing on teaching and learning is normally referred to as backwash
Synonymous term: washback (Buck (1988, p.17), Shohamy (1992, p.513), Bachman and Palmer (1996, pp.29-35), and so on).
Messick (1996, p.241): For a test to have positive washback, its tasks should be criterion samples – that is, “authentic and direct samples of the communicative behaviors of listening, speaking, reading and writing of the language being learnt”

8. Washback effects of a test
Bailey (1999): Washback process may concern:
Teachers
Students
Test developers
Teacher educators
Curriculum planners
Administrators
Language inspectors
Test users
Material developers and publishers
Parents

9. Chapter II: The Study
II.1. BACKGROUND TO THE STUDY: THE CONTEXT OF TEACHING AND TESTING ENGLISH LISTENING COMPREHENSION OF FIRST YEAR STUDENTS
II.1.1. Standard levels of mastery of Listening comprehension skill for first year students
II.1.2. An overview of the second-semester English listening program and its testing in Division 1 – English Department – CFL – VNUH
II The second-semester listening program in Division 1 - English Department – CFL
II The testing of listening comprehension for first year students in English Department - CFL

10. Chapter II: The Study II.2. METHODOLOGY OF THE STUDY
II.2.1. A qualitative and quantitative research
II.2.2. Subjects
II.2.3. Methods and procedures of data collection
II.2.4. Methods of data analysis

11. Chapter II: The Study II.2. METHODOLOGY OF THE STUDY
II.2.1. A qualitative and quantitative research
II.2.2. Subjects
II.2.3. Methods and procedures of data collection
II.2.4. Methods of data analysis
50 K41 students
12 teachers of Division 1 – English Department

12. Data for the study were collected from 6 sources:
Chapter II: The Study
II.2. METHODOLOGY OF THE STUDY
II.2.1. A qualitative and quantitative research
II.2.2. Subjects
II.2.3. Methods and procedures of data collection
II.2.4. Methods of data analysis
Data for the study were collected from 6 sources:
Questionnaire survey for K41 teachers;
Questionnaire survey for K41 students;
Class observations of two classes;
Interviews with two randomly selected teachers;
Interviews with two randomly selected students;
Scores from the second semester end-term listening test.

13. Chapter II: The Study II.2. METHODOLOGY OF THE STUDY
II.2.1. A qualitative and quantitative research
II.2.2. Subjects
II.2.3. Methods and procedures of data collection
II.2.4. Methods of data analysis
Two groups of data: teachers’ responses and students responses
Themes ascertained
Qualitative analysis: Text rendering (D’Arbon, 2008)
Quantitative analysis: Frequency, percentage, charts

14. Chapter III: Data Analysis, Discussion of findings and Recommendations
RESEARCH QUESTIONS
(1) What are the washback effects of the test on teachers’ teaching content, methodology, attitude and behavior? Are they positive or negative?
(2) What are the washback effects of the test-taking experience and test results on K41 students on their learning content, learning progress, self-image, motivation, learning attitudes and their relationship with teachers? Are they positive or negative?

15. Teaching content
Teachers’ responses
Fulfillment of class teaching to the listening syllabus
Types of activities carried out during class time
Teaching materials

16. Teaching content Teachers’ responses
Total commitment of listening teachers to the syllabus
Homework checking and In-class practice: essential parts of the lesson
Taught skills: as suggested in the syllabus
Test covering learnt knowledge and skills
Use of past test papers and IELTS practice tasks as teaching materials
Test mentioning and practicing: increase as the test came nearer

17. Teaching methodology
Teachers’ responses
Following stages of a lesson and procedure of carrying out the main tasks
Maintaining sufficient time for classroom activities
Being willing to adjust teaching methodology
Mentioning the end-term test to draw students’ attention to the lesson
Spending more time for test practice and feedback in the last weeks of the semester

18. Attitudes and behaviors
Teachers’ responses
Teachers’ anticipation of changes on their part when having a detailed report of students’ past test scores

19. Impacts of test-taking experience
Students’ responses
Students’ perception of task 2 in the test
Students’ decision on changing learning contents

20. Task 2 in the test
Students’ responses

21. Changes in learning contents
Students’ responses

22. The test result influences …
Impact of test result
Students’ responses
The test result influences …
LSG
ASG
HSG
Your learning attitudes 6 2 4
Your learning strategies 3
Your motivation for learning this skill 5
Your self-image
Your learning progress in this skill 1
Your relationship with teacher

23. Findings – Answer to question 1
Washback effects of the test on teachers:
Not “teaching-to-the-test”
A bit much of teachers’ talk about the test
 More evidences of positive washback effects than negative ones

24. Findings – Answer to question 2
Washback effects of the test on students: complicated
Development of learning strategies
Preparation of study plan
Desire to earn good mark
Evidence of “learning for the test”

25. Recommendations to teachers
Guiding students to effective use of previous test papers
Encouraging students to seek for fun in learning listening
Being more sensitive with the content of teacher talk in class
Using teaching materials wisely
Constructing item-analysis report

26. Recommendations to students
Taking it easy when dealing with the test
Using learning materials effectively
Developing the habit of analyzing weak points and making plans to improve them
Seeking for help and support from teachers and peers

27. Conclusion Comprehensive responses to research questions
Significant contribution to the deep evaluation of one criterion of a good test

28. THANK YOU FOR YOUR ATTENTION!!!

29. INTRODUCTION Rationale Aims and objectives Scope of the study
Research questions
Research methodology
Significance of the study

30. Aims and objectives
(1) Investigating the washback effects that the second end-term listening test has on teachers of listening skill for K41 students;
(2) Investigating the washback effects that the second end-term listening test has on K41 students of English department;
(3) Evaluating whether such evidences of washback (if any) are positive or negatives to teachers and students;
(4) Proposing ways to enhance the positive impacts and to minimize the negative effects of the test on teachers’ teaching and students’ learning.

31. Research methodology 50 students 12 teachers
Quantitative and qualitative approach
Data collection tools:
- Semi-structured interviews
- Questionnaires
- Classroom observations
Subjects:
50 students
12 teachers

32. Significance of the study
provides a reliable and profound background on the matter
helps the researcher gain more knowledge and skills in this field
offers an insight into test washback, an area rarely investigated in a university educational context
highlights the close relationship between teaching - learning and testing

33. Chapter I: Literature Review
I.1. Language testing in communicative approach
I.1.1. What is a language test?
I.1.2. Testing in Communicative Approach
I.1.3. Purposes of language testing
I.1.4. Washback effect of a test
I Washback: definitions and types
I Test-takers and washback
I Teachers and washback
I Other participants and washback

34. Language testing
Allen (1974, p.313): “a test is a measuring device which we use when we want to compare an individual with other individuals who belong to the same group.”
Heaton (1988, p.5): tests are considered first as “means of assessing the students’ performance” and then as devices to “motivate students.”

35. Communicative competence
Canale (1983, p.339, cited in Tran, 2002, p.5) proposes four dimensions of CC:
- Grammatical competence
- Sociolinguistic competence
- Discourse competence
- Strategic competence

36. Purposes of language testing
For teachers
evaluate
reveal teaching effectiveness
provide additional materials
For students
detect strong points and weak points
be a motivation

37. Chapter I: Literature Review
I.2. Achievement Language Test
I.2.1. Definition
I.2.2. Kinds of achievement tests

38. Chapter I: Literature Review
I.2. Achievement Language Test
I.2.1. Definition
I.2.2. Kinds of achievement tests
Hughes (2001, p.10): “…achievement tests are directly related to language courses, their purpose being to establish how successful individual students, groups of students, or the course themselves have been in achieving objectives.”

39. Chapter I: Literature Review
I.2. Achievement Language Test
I.2.1. Definition
I.2.2. Kinds of achievement tests
Two kinds of achievement tests:
- Progress achievement tests
- Final achievement tests

40. Chapter I: Literature Review
I.3. Testing listening comprehension skill
I.3.1. The nature and sub-skills of listening comprehension
I.3.2. Testing listening comprehension

41. Chapter I: Literature Review
I.3. Testing listening comprehension skill
I.3.1. The nature and sub-skills of listening comprehension
I.3.2. Testing listening comprehension
Traditional approach
Alternative approach
McDonough & Shaw (1993):
Processing sounds
Processing meaning

42. Chapter I: Literature Review
I.3. Testing listening comprehension skill
I.3.1. The nature and sub-skills of listening comprehension
I.3.2. Testing listening comprehension
Kitao and Kitao (1996, p.1):
Understanding sentences and dialogues: Interpreting meaning, responding to utterances
Task using visual materials: Matching, T/F, Mapping, Drawing
Tasks involving talks and lectures: Summary-filling, Matching, T/F, Short answers, MCQs, Note-taking, etc.

43. Motivation to learn listening skill
Students’ responses
Factor
Yes No 1
Necessary for future job 48
100% 2
Interest in English 40
83.3% 08
17.7% 3
Compulsory 18
37.5% 30
62.5%

44. Teaching methodology Teachers’ responses
Frequency of teachers following teaching process and conducting the main task in a procedure

45. Teaching methodology Teachers’ responses
Teachers’ reference to the end-term test to draw students’ attention

46. Students’ responses
* low-scored group (LSG): five students (<5 points) * average-scored group (ASG): 24 students (5 to 6 points) * high-scored group (HSG): 19 students (7 to 10 points:18 students of 7-8 points and 1 student with 9-10 points)

47. Frequency of activities in listening lessons
Students’ responses

48. Teaching methodology Process of lesson Reference to the end-term test
Teachers’ responses
Process of lesson
Reference to the end-term test

49. Students’ past test score
Teachers’ responses
Teachers’ care about students’ past test score
Teachers’ preference of having item-analysis report of students’ past test scores

50. Students’ in-class performance
Teachers’ responses
Interest in talks about test-taking techniques and test practice
High appreciation of test practice in-class
High appreciation of the importance of the end-term test

51. Classroom observation
Time: Week 13, 14, 15 of the second semester
Comments:
Maintenance of the process of a listening lesson
Fulfillment of requirements of the course outline.
Times of mentioning the end-term exam and discussing test-taking techniques
No exam pressure put on students.