CAA Options: Reading Collection of Evidence Lesley Klenk, CAA Options Administrator Fall Workshop October 2007

What is a Reading COE?  Reading is the process of making meaning of text. Students must be able to demonstrate their comprehension, their ability to analyze, and their critical thinking related to text in written responses.  The Reading COE is a set of classroom work samples created by a student that show what the student knows and can do in reading. They must demonstrate the same skills measured on the Reading WASL.

What is a Reading Collection of Evidence?  It demonstrates a breadth and depth of the WASL skills and knowledge in reading.  A collection must include 8 – 12 work samples that address all six strands. Each work sample must address at least two strands. Two work samples must be short papers, one literary and one informational.  Work samples may be on-demand or extended time.  On demand work is done in one sitting with teacher supervision.  Extended time work samples are student generated with limited teacher assistance.

What Does Breadth and Depth Look Like in a Reading COE?  The Reading COE demonstrates a breadth and depth of the WASL skills and knowledge in reading.  A collection is composed of written responses to a wide variety of texts. Those texts should be of high school rigor and may come from various content areas i.e. biology, U.S. History, business law, language arts.  The work sample will clearly state the strand being addressed (ie. literary comprehension) and include text based evidence that explains and supports the demonstration of that skill.

Scoring the Reading COE  239 Registered; 18 submitted; 15 met standard  71 points out of 96 points was judged to be proficient  65 points-70 points is the augmentation band  90 was the highest score and 54 was the lowest score  Too few collections to “set standard”. Instead, a Proficiency Committee studied Performance Level Descriptors for WASL and judged which collections were proficient.

Reading COE Sufficiency Issues  Work Samples that included the text the student read were insufficient and were not scored.  Work Samples that had grades on them were insufficient and were not scored.  Work Samples that did not state the text’s name in the assignment or the student response were insufficient and were not scored.

Reading COE Scoring Rules  Only sufficient collections are scored.  Each strand in each work sample is scored.  Each collection (all work samples) are scored twice.  The top two scores for each strand counts towards the total score.  96 points possible and 71 points judged sufficient.

How do the Reading COE scores add up? Collection # Scorer 1LCLALTICIAIT Total Points Points Possible Points Earned Scorer 2LCLALTICIAIT Total Points Points Possible Points Earned Total : Scorer 1 and Scorer

Strong Reading COEs…  had excellent assignments that explicitly asked students to demonstrate a specific reading skill.  featured texts that were interesting and timely to students.  demonstrated that students know how to cite text evidence to show understanding.  gave students “meaty” assignments that required more than just a few sentences to answer.  showed confident readers who weren’t perfect but showed definite strengths

Weak Reading COEs…  had hard to understand assignments with fact- oriented questions as opposed to skill-oriented ones.  featured texts that were technical without context; historical without background; or lean and lacking details.  used little to no textual evidence for support of claims about text.  had a preponderance of research papers where it was almost impossible to find the text much less the comprehension, analysis, or evaluation.  showed readers uncomfortable with manipulating text.

Lessons Learned  If students don’t know the skill going into a work sample, they don’t magically learn it in the process.  Students with weak COEs don’t know how to use textual evidence for support. In fact, they don’t include much support at all.  “Personal responses” did not lend themselves to target questions.  Several work samples attached to one text “lost steam.” Kids got tired of dredging up the same evidence.  Summer module assessments were moderately successful. However, they were rarely 4-point answers because the number of details limited their responses.

Scoring Notes  While scoring student work, it was easier to identify 3 or 4 from the rubric than 1 or 2 from the rubric.  Often, the Work Sample Documentation form was incorrectly filled out; the student lost points because the scorer had to score what was indicated.  Students did better when they turned in the maximum number of work samples; only the top two scores for a strand were reported.  Using OSPI generated tasks were easier to score— just because the tasks used the language of the targets.

Recommendations for February ‘08 and June ‘08 Scoring Windows Schools and districts that were most successful dedicated an elective class to working on COEs. Schools and districts that met regularly as a staff about the COE had more students meet standard. Schools and districts that created a calendar, a list of deliverables, and a set of goals had more students who met standard. Teachers who used OSPI tasks as samples created excellent tasks of their own. Educators who believed that students could meet standard saw it happen.