1. Essay assessors CPI 494, April 14, 2009 Kurt VanLehn

2. Analysis of ETS’ Criterion video demo

3. Enough essay topics available? Saw 20 for 8 th ? Fit of categories with standards? Fit with the teacher’s preferences How to insert graphics, videos, etc. into the essay questions?

4. Plan tools How to require? What if students choose the wrong plan? What about an explanation of each plan? Writer’s handbook? 10 min of planning sufficient? – Assuem that this is practice for standardized test? Conversion to paragraphs – OK? – How about compare & contrast or other non- linear plans?

5. Error reports ITS-wide feature: Good idea to present error categories that didn’t have any errors in this essay. Could encourage students X axis redundant Yes, clicking on category does show instances – Would be nice to see non-errors too

6. Repeated submission of an essay Does restricting the number of submissions prevents gaming? Does it compare submissions? Does it make you type them all in? Can instructors see submission history? Does it detect copying of essays when two students submit the same essay (even when they attend different schools). Similar essays?

7. Correcting errors Not easy to correct run on sentence Telling students to check handbook may not suffice Need a way to submit changes on a single sentence to check again. Need faster access to writer’s handbook. Maybe teacher is necessary for interpretation of general advice in specific context?

8. The organization & development tool How can it be used to improve the essay? – Only starts an analysis; doesn’t complete it. – Probably is saying, “according to the position in the essay, this should be the conclusion. Does it really read as a conclusion?” Student has to do most of the critiquing work How does it work with a longer essay? – What if essay is not a 5 paragraph essay? Does it pay any attention to the student’s plan? Can it provide a plan that comes from the essay?

9. Good ideas for all ITS (Most should be in an LMS) Inserting teacher comments in to the marked up solution. – May need a way to highlight a section of text – Library of comments Students can insert comments into a solution – And it gets flagged in the gradebook! – Should be tied in with LMS’s forums and email system – Threading Report the most common errors of students in a class so teacher can plan lessons Report generation for administrators e.g., portfolio management; accountability

10. Authoring tools Advice to authors about what kind of task to create E-writer needs a corpus of good and bad essays in order to calibrate it, so it will not work on an instructor’s essay question – until the instructor has hand-scored several hundred student essays The Critique Writing Analysis Tools are general and appear not to need such calibration, so they should work on the author’s essay questions

11. Answer based feedback? First sentence says its best for learning writing Is step-based feedback reasonable? How? – Can it be incorporated into the planning tools?

12. Word weight? Given a topic, probably means frequency(on-topic essays) / frequency(all essays, including off topic ones)

13. E-raters’ Topic vs. argument topic? What is the difference? Why have both?

14. Why a different regression for each essay question? Writing gendres?