Evaluating a Task-based English Course: A Proposed Model Pamararat Wiriyakarun

Background of the study A curriculum innovation at KMUTT The implementation of the task-based curriculum(strong form vs weak form) Concerns about the effectiveness of the curriculum Series of constant and informal changes An urgent need for a systematic and formal evaluation.

Objective of the study To evaluate a task-based English course provided by School of Liberal Arts, King Mongkut’s University of Technology Thonburi based on a set of criteria and the Integrated Stake-Tyler (IST) model proposed specially for this study.

Figure 1: Overview of course type in the English curriculum at KMUTT

LNG 102: the evaluated course Course title: Fundamental English II Type of course: a task-based English course Institute: King Mongkut’s University of Technology Thonburi Course concepts (or goals) : strategy training, developing cognitive skills and autonomous learning, and learning English for occupational and academic purposes. Course content: 3 main tasks: dictionary task, portfolio task and resourcing task 6 sub-tasks: using the dictionary, getting main idea and related details, note-taking from reading, and note-taking from listening summary writing, and presentation skills Assessment: (1) ongoing assessment, and (2) summative assessment

LNG 102: the evaluated course Course Description: This course builds on LNG 101 by providing further strategy training and places a strong emphasis on the cognitive skills of the students. A wide range of cognitive skills is covered including logical, creative and critical thinking. These cognitive skills are dealt with through a series of English medium tasks, which include problem solving, library tasks and Internet projects. Students’ language difficulties are dealt with as they arise thus addressing students’ real needs. Covering all four skills, the content of the course includes occupational as well as academic English. To enhance autonomous learning skills, students are encouraged to learn independently using the Self-Access Learning Centre as well as Internet and library resources.

Stake’s Comparison of Responsive Evaluation and preordinate evaluation Identifying issues, goals 10 Preparing instruments 30 15 Observing the program 5 Administrating tests, etc. -

Stake’s Comparison of Responsive Evaluation and preordinate evaluation (Continued) Gathering judgments - 15 Learning client needs, etc. 5 Processing formal data 25 Preparing informal reports 10 Preparing formal reports 20

The IST model Stake’s responsive evaluation Tyler’s objectives-based evaluation the researcher’s initiatives

Ultimate goals of Stake’responsive evaluation To evaluate what is really happening in the program rather than what we expect it would be like. To examine whether the evaluated program responds to audience requirements for information. To investigate different value perspectives of the people involved in reporting the success and failure of the program.

Tyler’s objectives-based evaluation Straightforward procedure of assessing learner achievement of objectives that determines success or failure and justify improvements, maintenance, or termination of program activities.

The researcher’s initiatives Portfolio assessment Self-assessment on autonomy in language learning

The principles underlying the entire evaluation process Negotiating with all parties of stakeholders about the issues to be evaluated. This stage is very important as it would help ‘respond’ their different interests. Involving as many of the stakeholders as possible throughout the evaluation process. Full cooperation from all stakeholders are required. Everyone concerned should understand the focus of this evaluation and provide input for the data-gathering process.

The principles underlying the entire evaluation process (continued) Authorizing students to act as ‘evaluators of their own learning’ in the evaluation process. In examining learning process, learners’ self-assessment is the most appropriate measure that can reveal how they view their own learning both in class and outside the classroom. Requiring feedback from the stakeholders as well as audience. Narrative reports of each phase of evaluation procedure should be submitted to the people involved to confirm or disconfirm the data. In so doing, the stakeholders may not perceive the evaluation as a threat, on the other hand, they may view that the reports are fair and reliable.

The principles underlying the entire evaluation process (continued) Refraining from introduce any changes. This evaluation aims at discovering the hidden truth of a particular curriculum, i.e. what is going on in the development and use of a certain curriculum. The evaluator does not initiate any change. It is the audience that will use the results of the evaluation in making judgement about the curriculum.

Evaluation Procedure Interview stakeholders and audience, overview program scope and activities, to identify purposes and issues. Identify data needs and selected instruments. Develop instruments. Validate the instruments by experts. Revise the instruments.

Evaluation Procedure (Continued) Conduct pilot study. Revise the instruments for the second time. Apply analytical procedures. Review course-related documents. Conduct task analysis and material evaluation. Analyze and interpret data. Prepare informal reports on the first part of the study and present them to the stakeholders and audiences to react to the data.

Evaluation Procedure (Continued) Do the main study. Analyze and interpret data. Prepare informal reports on the second part of the study and present them to the stakeholders and audience to react to the findings. Assembly reports and prepare a final report to the audiences.