1. What Really Goes on at an AP Statistics Reading? USCOTS May 18, 2007 Brad Hartlaub Kenyon College Daren Starnes Fountain Valley School of Colorado

2. The Growth Continues! YEAR EXAMS % PASSING 1997 7,667 62.2 199815,488 59.7 199925,240 57.1 200034,118 53.7 200141,609 59.7 200249,824 56.8 200358,230 61.9 200465,878 59.9 200576,786 60.7 200688,237 60.2 200796,139 ?

3. Grading the AP Exam  Rubrics (scoring standards) are drafted by the Chief Reader  The Test Development Committee reviews the rubrics and refines the free response questions  Rubrics are revised and the Test Development Committee reviews the rubrics for a second time (approximately 6 months after the first review)  The Chief Reader makes final revisions to the draft rubrics for the pre-reading meetings

4. Pre-Reading Meetings  Four days before the reading begins question teams composed of high school teachers and college and university professors arrive at the reading site (Louisville, Kentucky) to begin sample selection  Student samples are selected from hundreds of exam booklets to form training packets and revisions are made to the draft rubrics  The leadership team, which will consist of 75 leaders in 2007, conducts practice briefings (training sessions) and final decisions are made on scoring

5. A Typical Reading Day  A briefing is held to train the readers on how to score a particular question 445 readers will score approximately 100,000 exams in 2007 445 readers will score approximately 100,000 exams in 2007 Each reader will score at least two different questions during the seven day reading Each reader will score at least two different questions during the seven day reading  Readers go to reading rooms and work with a partner on one folder of exams (25 exams in a folder) and then begin scoring other folders  Two leaders are in each reading room to answer scoring questions and backread scored exams

6. AP Statistics Exam: Structure and Scoring  The AP Statistics Exam consists of two sections: 40 Multiple Choice Questions90 minutes counts 50% of exam score 6 Free Response Questions90 minutes 5 open-ended items @ 12 minutes 1 Investigative Task @ 30 minutes counts 50% of exam score   Purpose of the investigative task: to evaluate students’ understanding in several content areas and to assess their ability to integrate statistical ideas and apply them in a new context or in a nonroutine way.

7. Holistic Grading  We use a holistic scoring system that is probably very different from any system you have ever used to grade papers, projects, or exams: 4 Complete Response4 Complete Response 3 Substantial Response3 Substantial Response 2 Developing Response2 Developing Response 1 Minimal Response1 Minimal Response 0

8. Mock Briefing  Investigative Task from 2006  Handouts A Guide to Scoring Free-Response Statistics Questions A Guide to Scoring Free-Response Statistics Questions Question and Rubric (Scoring Standards) Question and Rubric (Scoring Standards) Student Samples Student Samples Scoring Commentary Scoring Commentary

9. Grade Setting  Each student receives a composite score (100 possible points – 50 for multiple choice section and 50 for the free response section)  Grades are set based on the distribution of the composite scores. 5 – Extremely well qualified5 – Extremely well qualified 4 – Well qualified4 – Well qualified 3 – Qualified3 – Qualified 2 – Possibly Qualified2 – Possibly Qualified 1 – No recommendation1 – No recommendation

10. Score Distributions for 2005 and 2006 Score 2005 Percent2006 Percent 512.512.5 512.512.5 422.822.3 422.822.3 325.225.4 325.225.4 219.318.3 219.318.3 120.221.5 120.221.5

11. Student Performance (over 5 years)

12. Scoring Statistics Question20052006 11.412.11 21.250.46 31.460.64 41.891.04 51.111.00 61.620.83

13. Professional and Social Events  Professional Night  Best Practices Night  Statistical Papers Night  Social Lounges and Activities  Question Debriefings  Worst Practices Night

14. Professional Development Opportunity  Please consider getting more involved in the AP Statistics Program by becoming a reader.  We have been unable to hit our target of 60% college and 40% school readers.  To complete the online application, just point your web browser to AP Central (apcentral.collegeboard.com)

15. Question and Answer Session  Curricular Issues  Course Content  Statistical Software  Course Projects  Post-Exam Activities  Test Development  The Exam  Rubrics  The Reading  Grade Setting