IEP/HE SIG Forum The TOEFL iBT: Three Years Later CoTESOL “Welcoming the World” October 23, 2008

 Reading

 Listening

 Reading  Listening  Break

 Reading  Listening  Break  Speaking

 Reading  Listening  Break  Speaking  Writing

 Reading  Listening  Break  Speaking  Writing Total test time: 4 hours

 Reading: 3-5 passages, 20 minutes each  Listening  Break  Speaking  Writing

 Reading: 3-5 passages, 20 minutes each  Listening: 6-9 passages, minutes total  Break  Speaking  Writing

 Reading: 3-5 passages, 20 minutes each  Listening: 6-9 passages, minutes total  Break: 10 minutes  Speaking  Writing

 Reading: 3-5 passages, 20 minutes each  Listening: 6-9 passages, minutes total  Break: 10 minutes  Speaking: 6 questions, second responses  Writing

 Reading: 3-5 passages, 20 minutes each  Listening: 6-9 passages, minutes total  Break: 10 minutes  Speaking: 6 questions, second responses  Writing: 2 questions, 50 minutes total

PBT CBT iBT

1. Does ETS recommend minimum TOEFL iBT scores for students entering undergraduate and graduate programs? If not, how do universities choose minimum entry scores for the TOEFL iBT?

No, we don’t. Because the needs of different universities and programs are so varied, ETS does not recommend entry scores, but provides information to institutions in setting scores for their particular circumstances.

Here is a link to the information on the ETS website: ecfd5b8849a77b13bc /?vgnex toid= cb110VgnVCM f95 190RCRD&vgnextchannel=ee76d0b941abb11 0VgnVCM f95190RCRD ecfd5b8849a77b13bc /?vgnex toid= cb110VgnVCM f95 190RCRD&vgnextchannel=ee76d0b941abb11 0VgnVCM f95190RCRD

At the highest level, ETS offers institutions 3 alternative methods of setting scores:

 Use the TOEFL iBT standard-setting CD-ROM. E- mail or call to obtain the

 Use the TOEFL Score Comparison Tables to find which TOEFL iBT scores are comparable to your score requirements on TOEFL PBT. We also encourage universities to look at the annual TOEFL test and Score Data Summary which includes student performance on the test by native language, native country, gender and purpose for taking the test as well as percentile data. This can be very helpful when comparing iBT score requirements with previous CBT or PBT requirements.

Score Data Summary: c988ba0e5dd572bada20bc47c /?vgne xtoid=20beaf5e44df4010VgnVCM f9 5190RCRD&vgnextchannel=d35ed898c84f40 10VgnVCM f95190RCRD

OR  Go to  Select "TOEFL Research" on the left hand side of the screen

 Consider the score requirements set by peer institutions.

2. Are universities satisfied with the score setting or do they think it needs some adjustment with consideration given to the different sections?

We are encouraging universities to review their score requirements now that students have been admitted with TOEFL iBT scores, and ETS had carried out academic studies, so there is actual test taker performance data in the annual Test and Score Data Summary. This is good score use practice and we support it.

3. What type of feedback is ETS receiving from ESL professionals, universities, and students on the TOEFL iBT?

Overall, the feedback is quite positive. In the beginning, we had some problems getting enough seats to meet the demand, but that problem is easing now. We get positive feedback about some of the changes we have made in TOEFL iBT, such as adding the speaking measure, adding integrated items, eliminating discrete grammar items, and allowing note taking.

Here is a link to information on our website, with further links to news stories about TOEFL iBT: a8b4042d69a77b13bc /?vgnextoid= 811ed898c84f4010VgnVCM f95190RCR D a8b4042d69a77b13bc /?vgnextoid= 811ed898c84f4010VgnVCM f95190RCR D Some market research we recently conducted brought us the complaint from a student that you actually had to learn English to pass TOEFL, while with other tests you could just do test prep. We took that as a compliment!

4. What percentage of test takers challenge their scores? Does ETS have a percentage for successful challenges?

Sorry, this is proprietary information that we don’t publish.

Here is information on how to apply for a rescore: ecfd5b8849a77b13bc /?vgnex toid=1b01fa VgnVCM f95 190RCRD&vgnextchannel=c025a1e13bf36110 VgnVCM f95190RCRD

OR  Go to  Select "Test Scores"  Select "Internet-based Test"  Pick the tab that says "Cancel, Reinstate, Rescore"

5. When are the PBT and CBT going to be phased out in the U.S? Are there plans to phase out these tests abroad?

The CBT test ended in July of 2006 and because scores are valid for 2 years, institutions should not be receiving CBT scores any longer. So CBT for TOEFL is now history. 95% of all TOEFL testing today is Internet- based.

The PBT test is currently offered in countries without the infrastructure to support the iBT and for the foreseeable future we will need to continue with it. We monitor this situation regularly because our goal is to bring TOEFL iBT to those countries as quickly as possible. Until we can, we’ll continue to offer the paper-based TOEFL, in order to serve students in those locations.

In addition, we used the paper-based TOEFL in a very limited way to support the TOEFL iBT test center network. Now that the TOEFL iBT testing network has matured, 3 years into the process, we don’t use it very much since 95% of all our testing is Internet-based. However, students can still be admitted to a university with a PBT score.

The PBT is still offered in the U.S. 6 times per year in cities such as San Francisco, New York, and Boston. You can go to ETS’ website and search for PBT sites by clicking on each state individually and searching for “paper-based test” on the list of testing centers: toefl

6. Have you had native English speakers take the TOEFL iBT? If so, does ETS have any data how native speakers perform on the test?

Yes, we have administered TOEFL to native speakers, but the report has not been published yet. There is a published research report on the performance of native speakers on CBT, available in PDF format at: c988ba0e5dd572bada20bc47c /?vgne xtoid=43b1af5e44df4010VgnVCM f9 5190RCRD&vgnextchannel=d35ed898c84f40 10VgnVCM f95190RCRD c988ba0e5dd572bada20bc47c /?vgne xtoid=43b1af5e44df4010VgnVCM f9 5190RCRD&vgnextchannel=d35ed898c84f40 10VgnVCM f95190RCRD

Some high level findings in the research on the iBT and native speakers (though not published yet) were:  Overall, native speakers performed better on the TOEFL than non-native speakers, but did not “ace” it as a group;  The Reading and Writing Sections showed moderate differences between native speakers and non-native speakers;  The Listening section showed greater differences between native speakers and non-native speakers;  The Speaking section showed the greatest differences between the native speakers and non-native speakers. The lowest score on the rubric for Speaking was rarely given to non-native speakers.

The results of this study are still being analyzed and the report being written, but research reports are put on the ETS website in PDF format when they are published, so those who want more detail could check there periodically for this report:

If you are interested in ETS’ ongoing research for TOEFL, here’s a link for a framework of recent research. You’ll see it’s on a variety of topics to ensure test quality. mework_Recent_TOEFL_Research.pdf mework_Recent_TOEFL_Research.pdf

7. How successful does ETS consider the TOEFL iBT? What are some challenges ETS faces with this new format?

We’re very pleased with the TOEFL iBT test. Overall test reliability ranges between.94 and.95 and the new integrated questions are performing as we expected in terms of discriminating those with greater or lesser proficiency. The new integrated questions and the way TOEFL does scoring for the Speaking and Writing sections are what we are probably most proud of.

All test results, including the spoken and written responses, are sent to ETS via the Internet after the test and are then distributed to our online scoring network for scoring. Each written response is scored by two raters and spoken responses are distributed to between 3-6 different scorers. ETS separates the testing and scoring environments to maintain security, validity and reliability. Thus, there is no possibility the scorer may know the test taker, and scorers evaluate worldwide responses.

Some of the challenges we have faced with TOEFL iBT are:

 Building a test center network for TOEFL iBT. For CBT we had 617 test centers. Today we have grown to more than Recruiting, training and supporting that number of secure test centers to ensure we had sufficient test center space required great effort. Three years on, the network is now mature and we’ve managed to move beyond the initial learning curves.

 Ensuring we have sufficient raters. On TOEFL CBT, test takers were asked to write one essay and there was no speaking section. Now we need scores for 2 essays and 6 spoken responses for each student.

 Reliable electricity supply. As mentioned earlier, we need to have a reliable electrical supply in- country and at least intermittent Internet supply to offer the iBT test.

 Helping universities and other score users understand TOEFL iBT scores and use them well. We’ve spent a great deal of effort helping score users understand the new score scale, how to set new score requirements, and to have good practices regarding using TOEFL scores. That includes looking for other evidence of English proficiency in the application. A TOEFL score provides very useful information but it is not the full story. We’ve encouraged universities to think beyond setting a total score requirement, and to consider section score information as well.

1. What are the most common errors students make in the Writing section? What can teachers do to help students improve their responses to these tasks?

 Most important is that students understand that on the integrated task they are not being asked for their opinions. They should be working with the information given.  For "errors", there is absolutely nothing ETS could zero in on. The independent essay task is little changed from the original TWE and assesses degrees of competence in language, development and organization.

2. In practicing for the test, would it be more effective for students to write summaries of what they read and heard, or should they practice writing about how the points of view compare?

They could practice a variety of approaches to see what is easiest/most effective for them.  They could summarize reading points one by one and follow each point with information about how the lecture point relates.  They could summarize the lecture points one by one and follow each point by how it responds to the reading point.  They could briefly summarize the entire reading and then summarize the lecture being sure that their summarization includes how the lecture responds. All of these approaches are quite valid.

Initially, ETS planned to introduced a variety of accents, but through a pilot test, they found that students performed worse when there was variety of accents compared to a single, standard American accent.

These slides are available on CoTESOL’s new wiki at As the forum progresses, if there are any questions the participants cannot answer, we will send them into ETS for answers. These answers will also be posted on CoTESOL’s wiki.

Minimum TOEFL Scores Required Undergraduate Majors Engineering majors560 (220*)(83**) All other majors500 (173*)(61**) * Equivalent score based on the computer-administered TOEFL. ** Scores on the internet based TOEFL.

Minimum TOEFL Scores Required Graduate Programs Aerospace engineering 560 (220*) (83**) Biological sciences — ecology and evolutionary biology 550 (213*) (79**) Biological sciences — molecular, cellular, and developmental biology 600 (250*) (100**) Business (doctoral) 620 (260*) (105**) Chemical Engineering 600 (250*) (100**) Chemistry 600 (250*) (100**)

Civil engineering 530 (200*) (71**) Comparative literature 550 (213*) (79**) Computer science 560 (220*) (83**) Dance 550 (213*) (79**) Economics 525 (197*) (71**) Education 650 (280*) (114**) Electrical engineering 600 (250*) (100**)

Journalism 550 (213*) (79**) Linguistics 550 (213*) (79**) MBA 600 (250*) (100**) Mechanical engineering (master's) 550 (213*) (79**) Mechanical engineering (doctoral) 600 (250*) (100**) Music (doctoral) 550 (213*) (79**) Physics 575 (233*) (89**)

Spanish 550 (213*) (79**) Telecommunications 600 (250*) (100**) Theater 550 (213*) (79**) All others 500 (173*) (61**) *Equivalent score based on the computer-administered TOEFL. ** Score on the internet based TOEFL.