What Really Goes on at an AP Statistics Reading? USCOTS May 18, 2007 Brad Hartlaub Kenyon College Daren Starnes Fountain Valley School of Colorado

The Growth Continues! YEAR EXAMS % PASSING , , , , , , , , , , ,139 ?

Grading the AP Exam  Rubrics (scoring standards) are drafted by the Chief Reader  The Test Development Committee reviews the rubrics and refines the free response questions  Rubrics are revised and the Test Development Committee reviews the rubrics for a second time (approximately 6 months after the first review)  The Chief Reader makes final revisions to the draft rubrics for the pre-reading meetings

Pre-Reading Meetings  Four days before the reading begins question teams composed of high school teachers and college and university professors arrive at the reading site (Louisville, Kentucky) to begin sample selection  Student samples are selected from hundreds of exam booklets to form training packets and revisions are made to the draft rubrics  The leadership team, which will consist of 75 leaders in 2007, conducts practice briefings (training sessions) and final decisions are made on scoring

A Typical Reading Day  A briefing is held to train the readers on how to score a particular question 445 readers will score approximately 100,000 exams in readers will score approximately 100,000 exams in 2007 Each reader will score at least two different questions during the seven day reading Each reader will score at least two different questions during the seven day reading  Readers go to reading rooms and work with a partner on one folder of exams (25 exams in a folder) and then begin scoring other folders  Two leaders are in each reading room to answer scoring questions and backread scored exams

AP Statistics Exam: Structure and Scoring  The AP Statistics Exam consists of two sections: 40 Multiple Choice Questions90 minutes counts 50% of exam score 6 Free Response Questions90 minutes 5 open-ended 12 minutes 1 Investigative 30 minutes counts 50% of exam score   Purpose of the investigative task: to evaluate students’ understanding in several content areas and to assess their ability to integrate statistical ideas and apply them in a new context or in a nonroutine way.

Holistic Grading  We use a holistic scoring system that is probably very different from any system you have ever used to grade papers, projects, or exams: 4 Complete Response4 Complete Response 3 Substantial Response3 Substantial Response 2 Developing Response2 Developing Response 1 Minimal Response1 Minimal Response 0

Mock Briefing  Investigative Task from 2006  Handouts A Guide to Scoring Free-Response Statistics Questions A Guide to Scoring Free-Response Statistics Questions Question and Rubric (Scoring Standards) Question and Rubric (Scoring Standards) Student Samples Student Samples Scoring Commentary Scoring Commentary

Grade Setting  Each student receives a composite score (100 possible points – 50 for multiple choice section and 50 for the free response section)  Grades are set based on the distribution of the composite scores. 5 – Extremely well qualified5 – Extremely well qualified 4 – Well qualified4 – Well qualified 3 – Qualified3 – Qualified 2 – Possibly Qualified2 – Possibly Qualified 1 – No recommendation1 – No recommendation

Score Distributions for 2005 and 2006 Score 2005 Percent2006 Percent

Student Performance (over 5 years)

Scoring Statistics Question

Professional and Social Events  Professional Night  Best Practices Night  Statistical Papers Night  Social Lounges and Activities  Question Debriefings  Worst Practices Night

Professional Development Opportunity  Please consider getting more involved in the AP Statistics Program by becoming a reader.  We have been unable to hit our target of 60% college and 40% school readers.  To complete the online application, just point your web browser to AP Central (apcentral.collegeboard.com)

Question and Answer Session  Curricular Issues  Course Content  Statistical Software  Course Projects  Post-Exam Activities  Test Development  The Exam  Rubrics  The Reading  Grade Setting