An English Proficiency Test for Today’s Student Using Today’s Technology Marcie Mealia,

2 PTE Academic Launched in 2009 Developed to meet the need for a test that more accurately measures the communicative skills of international students in an academic environment as securely and efficiently as possible Accepted by over 3,000 programs worldwide Available in 50+ countries and 230+ locations Uses automated scoring technology

3 Automated Scoring Technology An efficient, quick process while providing consistent and objective results Ensures consistent item presentation across time, people, and locations Enables efficiency, objectivity, fairness, and consistency Dependable decision-making process

Development of the Data Over 18 months, test data were collected from more than 10,000 test takers from 38 cities in 21 countries who participated in PTE Academic’s field tests. These test takers came from 158 different countries and spoke 126 different native languages, including (but not limited to) Cantonese, French, Gujarati, Hebrew, Hindi, Indonesian, Japanese, Korean, Mandarin, Marathi, Polish, Spanish, Urdu, Vietnamese, Tamil, Telugu, Thai and Turkish. The data from the field test were used to train the automated scoring engines for both the written and spoken PTE Academic items. Presentation Title runs here l 00/00/004

5 How Automated Scoring Technology Works Speaking Sections (Ordinate Technology) PTE Academic Database Local Computer / Server Scoring Administrator’s data management “Vancouver is famous for its ethnic diversity.”

6 Automated Scoring Technology Assessed skills in Spoken scoring Sentence Mastery Content Vocabulary Accuracy Pronunciation Intonation Fluency Expressiveness Pragmatics

Intelligent Essay Essayer (IEA) Latent Semantic Analysis (LSA) Test takers are asked to write 200–300 word essays and 50– 70 word summaries. Response is submitted for scoring System evaluates the meaning of the response, as well as mechanical aspects of the writing. The system compares the response with the large set of training responses, computes similarities, and assigns a score based on content, The system assesses each trait (grammar, structure and coherence, etc.,) in the test taker’s response, compares it with the large set of training responses, and then ranks the response according to that trait. Presentation Title runs here l 00/00/007

8 How Automated Scoring Technology Works Writing Sections For a specific prompt, LSA engine is trained on 200+ essays and scored by expert human raters LSA Training Vector- based Comparison Score of the most similar essay is used to predict what a human score would be by using modern regression and classification methods Various variables added Coherence When a new essay is submitted, it is compared with all 200+ pre-scored essays Word Flow Word Choice

9 Automated Scoring Technology Assessed skills in Written scoring Word choice Grammar & Mechanics Progression of ideas Organization Style, Tone Paragraph structure Development, Coherence Point of view Task completion

10 Automated Scoring Technology Reliability and Validity LSA is as accurate as skilled human graders Agrees with a single human reader as often as two human raters agree with each other MeasureLSA to human raters Human raters to human raters Mean exact score agreement % % - 87 Mean exact + adjacent agreement % % - 100

Presentation Title runs here l 00/00/0011

Presentation Title runs here l 00/00/0012 Test Taker Process Before test day Book test online or over phone – Prompted to set up account in Pearson VUE – Test dates can be made available based on demand Prepare for PTE Academic – Courses – Scored Practice Test – Sample Questions – Free materials including: ○ Test tutorial ○ Skills video ○ Top tips ○ Offline practice test

Presentation Title runs here l 00/00/0013 Test Taker Process On test day Arrive to test center 30 minutes prior to test Show your ID and sign in digitally Get your digital photograph taken Scan your palm digitally – Must scan palm each and every time a test takers leaves or enters testing room Check all belongings into locker One single 3 hour sitting

Presentation Title runs here l 00/00/0014 Test Taker Process After test day Receive notification once your scores are available (within 5 business days) Login to your Pearson VUE account Assign score report(s) to institutions online through secure server – Ability to send unlimited number of score assignments at no extra cost – Institutions do not accept a paper or pdf copy of score report

Presentation Title runs here l 00/00/0015 PTE Academic Score Report Delivered through secure, computer-based server Scored against Global Scale of English – Granular score scale from Comprised of both communicative and enabling skills

Presentation Title runs here l 00/00/0016 PTE Academic Access Available in 50+ countries in 230+ locations Test sessions run 360 days per year around the world 51 new test centers opened in 2014

Presentation Title runs here l 00/00/0017 Test Taker Benefits A fast, flexible, and convenient process Objective and accurate results Results available within 5 business days (typically 48 hours) One single 3 hour sitting Available in 50+ countries worldwide Test sessions can be made available based on demand Provides most detailed score report compared to any other English proficiency test Ability to send unlimited score report assignments

Presentation Title runs here l 00/00/0018 “Once I had taken my test, I received my results within 48 hours; a very fast service which I greatly appreciated.” - Mr. Muhammad Saleem, UK, August 2014 “People tend to be more biased for [other tests]. They tend to judge you sometimes. You don't get the same score twice, so that way I think a computer based test is more fair and more true to your capabilities.” – Test taker interviews, February 2014 “It’s easier for me as I use computers a lot and I am confident, instead of having an examiner in front of me and asking me questions, which makes me nervous.” - Mr. Muhammad Nazim Rasheed, UK, August 2014

Presentation Title runs here l 00/00/0019