1. Ian Lucas Executive Director ETS Europe UK CRITERION ® Online Writing Evaluation

2. Criterion... is a web-based service that utilises automated scoring technologies to evaluate writing samples and provide students with an instant holistic score and annotated diagnostic feedback that analyses elements of grammar, usage, mechanics, style, and organisation and development.

3. CRITERION Criterion gives students valuable self- paced writing practice, diagnostic feedback and provides instant holistic scores on their essay writing performance - whether in the classroom or at home.

4. How Criterion Works Technology developed using scored student writing, rubrics, and linguistic and discourse analysis tools Holistic score feedback based on analysing syntax, organisation of ideas, and vocabulary Diagnostic feedback on errors in student writing through a linguistic model that identifies grammatical, mechanical, and usage errors

5. How Criterion Was Developed Technology derived from ETS research on natural language processing and information retrieval Essays scored by trained faculty readers 465 samples across the range of score points per topic Since 1999, 1.5 million GMAT essays have been scored with over 98% agreement with human scorers Writing analysis tools developed at ETS provide diagnostic feedback

6. CRITERION SUPPORTS TEACHING Offers the opportunity to practice writing Analyse essay development Customise topics & writing options Provides instant diagnostic feedback Summary reports of student performance Easy access to class performance

7. CRITERION SUPPORTS LEARNING Real-time in a Web-based application Use the writer’s handbook Revise essays based on review & feedback Opportunities to work from home Feedback on: Grammar Usage Mechanics Style Organisation & Development

8. What Criterion Can Provide A self-paced, risk-free learning environment Immediate (within 15 seconds) evaluation on writing performance Holistic score and diagnostic feedback Library of topics and sample essays for student and instructor review Individual, interactive student writing portfolios Intuitive Writer’s Handbook

9. What Criterion Cannot Do Criterion technology does not actually read essays; it uses natural language processing techniques Criterion is not perfect; it sometimes makes mistakes scoring essays Holistic scoring technology can be fooled Diagnostic feedback makes mistakes and does not catch all errors Criterion should not replace an instructor

10. Copyright © 2003 by Educational Testing Service. All rights reserved. UNTELE 2004

31. Copyright © 2002 ETS Technologies, Inc. All rights reserved. Criterion, the Criterion logo, ETS Technologies and the ETS Technologies logo are trademarks of ETS Technologies, Inc. UNTELE 2004

32. Copyright © 2003 by Educational Testing Service. All rights reserved. UNTELE 2004

37. Criterion... is a web-based service that utilises automated scoring technologies to evaluate writing samples and provide students with an instant holistic score and annotated diagnostic feedback that analyses elements of grammar, usage, mechanics, style, and organisation and development.

38. Thank you More information http://www.ets.org/criterion/tour Extra information admin@etseuropeuk.org ilucas@ets-europe.org