Beginning Scoring Presented by: Add your names here ( obvious, I know)

Goals for Scoring Sessions Learn to reliably score student work using the Scoring Guide. Understand that knowledge of the scoring guide is needed to elicit scorable student work. Improve confidence and speed at scoring student work. Explore value of student feedback. Have fun and learn from each other! Read these and likely you'll want to emphasize the challenge and practical need for bullet #3

A Science Challenge! What do you know about straws and how you can use them to move objects? We claim: “Characteristics of the straw will control how far you can blow the cotton ball” Carry out a quick investigation (10 minutes) As a team, report your findings to the group. Based on number of participants, group them into groups of 2-4 and set them to this task. Resist the urge to over-prescribe what they should do...rather encourage them to investigate as they see fit. use poster paper to illustrate, a couple of felt pens and have them report on those and post around the room. No oral reports unless someone has a burning need to do so.

Assessing Your Inquiry Fantastic! Good So-so Needs work Embarrassing here's a crude set of scores for them to score each piece of work on. Don't necessarily have each person report scores for each piece of work, but tlet them score ( 5 minutes) and then ask if there were any frustrations or problems with this scoring continuum.

Enter the Scientific Inquiry Scoring Guide… It defines the important aspects of a task, provides clear assessment guidance, and promotes uniformity of assessment and feedback to students/teachers. Ideally, participants will identify that the previous scoring task lacked any clarity or uniformity....The scoring adjectives weren't helpful, even tho there were 5 score levels to choose from. So a guide, a defined framework is required.....enter the scoring guide.

Oregon's Science Standards Were developed to include the following… Science content mastery as assessed with a standardized knowledge and skills test Science process which must be experienced and practiced. Process (via work sample) is assessed using the scoring guide Briefly, lets review how Oregon got to this place in science asessment in the first place. We are one of VERY FEW states that explicitly recognizes that science content and process are equally important and cannot be assessed in the same way!

What are the scoring dimensions? Forming a Question or Hypothesis Designing an Investigation Collecting and Presenting Data Analyzing and Interpreting Results Can you build the scoring guide?.. Puzzle Activity Review the pieces of the guide and then hand out the puzzle. Have each participant try to contruct the scoring guide, without referring to any copies of the guide that they might have.

A, N, C Pattern… You will notice that in each dimension and for each score there are three descriptors...labeled as A, N, and C...next slide  

A,N,C…The Mysterious “Threads” What do the “Threads” represent? Why were they created? Which are more important? Challence students to sort out what A, N and C are ( at table groups can you describe them?

A, N, C… A: Application of Scientific Knowledge N: Nature of Scientific Inquiry C: Communication Discuss the nature of these and why we think they are important: A....what do you know of the science subject? N....what do you know about the process of inquiry? C...how well can you communicate?

Scientific Inquiry Scoring Scale 6 5 4 3 Developing 2 Emerging Exemplary 6 Strong 5 Proficient 4 3 Developing Scoring scale...aknowledge that pass/fail is Most important, but that it's critical to have room above and below to give feedback. Also well established in assessment literature that a 6-point scale makes it easier for reliable, repeatable scoring. 2 Emerging 1 Beginning

Scientific Inquiry Key Distinctions What does a “4” paper look like? What differentiates a 3 and a 4? OK, what were we thinking here? Are we handing out a fully scored "4" paper? Or are we looking at the scoring guide only here? Now, it IS important here to recognize that the 3/4 distinction is the highest stakes decision that a scorer can reach, and we anticipate that papers that are right at the 3/4 split will takes us the most time to score well.

Important Issues! What do we mean by “preponderance of evidence?” Can evidence from throughout the work be used to score each dimension? Did you know one work sample is required per year beginning in 4th grade? Phase In Schedule and Work Sample Guidelines and FAQ documents are located in Science Teaching & Learning to Standards http://www.ode.state.or.us/tls/science/ Discuss these as necessary

OK, OK Let’s do some scoring already!……But first, some rules. Rules of the Road for scoring… We are not here to change the guide. We are not here to dispute the anchor papers or the tasks. We ARE here to understand that experienced teachers have reached scoring consensus. We ARE here to calibrate our scoring. Read, and emphasize that adhereing to these rules will make things go faster and more productively within what we are trying to accomplish today. Changes to the guide, development of new anchor papers are all possible, and ODE contacts can help with that, but they are beyond the scope of today's work.

Scoring the First Anchor Paper! This Anchor Paper meets the Standards in each dimension Why this score? Make notes and discuss with a partner. hand out Anchor paper#1, with at least all 4's. Have teachers read and discuss why it is considered a 4. Discuss with the group.

Lunch break likely here

Scoring to Improve Student Success: Words of Wisdom Assessment is only truly successful when results are used to improve instruction for individual students. -Johnson, 1987

Remember the rules… Rules of the Road for scoring… We are not here to change the guide. We are not here to dispute the anchor papers or the tasks. We ARE here to understand that experienced teachers have reached scoring consensus. We ARE here to calibrate our scoring. Just to reemphasize... It you feel it's too soon, just skip over it.

Anchor Scoring … Remaining Anchor papers Score each paper (whole numbers…be brave…and don’t peek!) Compare at table, reach consensus? Tally scores/reveal anchor scores Be sure to align yourself with consensus scores. Score remaining papers as time permits ( without anchor scores available)...encourage tables to discuss/debate scores and have them use the form on the next slide to score ( blanks should be in their packets). Reveal anchor scores as appropriate and be prepared to discuss at least briefly. Don't let yourself get dragged into lots of disagreements over anchor scores, tho.

Scientific Inquiry Documentation Sheet

Out on the Edge…..Scoring New Work Candidate anchor papers Score individually, documenting on forms provided. Please include score, and Why Not score above and below (see example handout). Feel free to discuss/debate, but please fill out score forms for each paper scored…the more the better Turn attention to candidate anchor paper packets. Use the justification forms provided to score. Hand out an example justification form so they can see how to fill one out.

Another break may be in order Another break may be in order. Gather up score forms for the candidate papers.

Scientific Inquiry in Your Classroom Which dimensions do you already spend the most time doing in the classroom? At this point we turn our attention to student feedback by asking the teachers to reflect a bit on their strengths in the classroom. recognizing that good inquiry must be taught, what of the 4 dimensions do they spend most of the time on and why....discussion that will get them focused back on their classroom. May spark a discussion of what's hard and easy to teach....If so, it would be good to point out that most find the analysis most difficult and that's why we are devoting this Leaders conference to that topic.

Formative Assessment and the Scoring Guide Scoring Guide is intended to be more than a final assessment tool. Teachers and Students both can improve inquiry skills through use of the scoring guide. Whatever our strengths as teachers of inquiry, we can make use fo the scoring guide as a feedback tool as well......is feedback important? See next slide.

Research suggests FEEDBACK is MORE important than grades. Value of Feedback Research suggests FEEDBACK is MORE important than grades.

Research Basis of Formative Assessment Writing assignment with students using well-known scoring criteria Three treatments - Students received Grades alone Grades + feedback Feedback alone Performance improved only in group that received feedback alone! ( Butler,R. 1987 and 1988) This was based on an extensive study out of Texas relating to a writing scoring guide well known to students. We report this only because is seems counterintuitive that feedback+grades isn't the best. The study hasn't been repeated, and never in science, but it provides at least intriguing evidence that feedback is VERY important in scoring-guide based assessments

The Role of Formative Assessment in Inquiry Student work Scoring guide Scores Feedback (For state/district) (For students/teachers) Simple graphic suggesting that feedback and scores are equally important.

Formative Assessment Process Targeted Oregon Science Standards Student Inquiry Work ID gaps between standards and student work Classroom Instruction? Student Performance? What should we do? What should students do? Another simple graphic that sets out a flow of assessment/info that we derive from the process. Emphasize that our work to ID the gaps between student work and expectations can lead us to find gaps in our instruction as well as gaps in student understanding and that we have to be open to both.

Feedback Issues Time to score PLUS time to generate feedback Who provides feedback ( can students help each other)? Forms of feedback ( T-charts, custom comment forms, conversations, peer editing) Discussion points about feedback, discuss as appropriate/as time allows.

Learning from student work… What do we tell the student? What do we learn as the teacher? How does this inform our instruction? Examples and Ideas Solicit examples and ideas from audience. Discuss as time allows

The Journey Continues For more information contact Aaron Persons, Office of Assessment and Evaluation, Oregon Department of Education aaron.persons@state.or.us or 503-378-3600 Ext. 2317