Dr. Robert Mayes University of Wyoming Science and Mathematics Teaching Center

Assessor – 3 basic questions What kind of evidence do we need to support the attainment of goals? Tasks that reveal understanding, such as comparing and contrasting or summarizing key concepts What specific characteristics in student responses, products, or performances should we examine to determine the extent to which the desired results were achieved? Criteria, rubrics, and exemplars are needed Does the proposed evidence enable us to infer a student’s knowledge, skill, or understanding? Validity and reliability concerns

Stage 2: Evidence Think like an assessor not an activity designer What should be sufficient and revealing evidence of understanding? What would be interesting and engaging activities on this topic? What performance tasks must anchor the unit and focus the instructional work? What resources and materials are available on this topic? Against what criteria will I distinguish work? How will I give students a grade and justify it to parents? Assessor Activity Designer

Stage 2: Evidence Think like an assessor not an activity designer Assessor Activity Designer How will I be able to distinguish between those who really understand and those who don’t (though they seem to)? What will students be doing in and out of class? What assignments will be given? What misunderstandings are likely? How will I check for those? Did the activities work? Why or why not?

Continuum of Assessment Methods Vary in several characteristics Scope: from simple to complex Time Frame: short-term to long term Setting: decontextualized to authentic Structure: highly structured to ill-structured Move from snapshot to scrapbook Self-assessment of sources of evidence (HO) Informal checks Observation/ Dialogue Quiz/ Test Academic Prompt Performance Task

Collecting a Range of Evidence Activity: (HO) determine a range of assessment evidence you may use related to the Enduring understanding Topics important to know and do Worth being familiar with Which assessment methods best fit the 3 categories? Worth being familiar with Important to know and do Enduring Understanding

Academic Prompt Assessments Open-ended question or problem that require student to prepare a specific academic response Think critically and prepare response Require constructed response under exam conditions Divergent – no single best answer Subjective judgment based scoring using criteria or rubric May or may not be secure Often ill-structured – require development of strategy Involve analysis, synthesis, and evaluation

Performance Task Assessments Complex challenges that mirror the issues and problems faced by adults Real or simulated settings, authentic Require student to address audience in non-exam conditions Divergent – no single best answer Subjective judgment based scoring using criteria or rubric, Greater opportunity to personalize task Not secure – students given criteria in advance

Performance Task – 6 Facets Activity: Use the 6 Facets of Understanding to generate a performance task related to your enduring understanding Questioning for Understanding (HO) Performance Verbs (HO) Performance Task creation (HO) Performance Task brainstorming (HO)

Performance Task -GRASPS Creating a performance task with context and roles Goal Role Audience Situation Product, Performance, and Purpose Standards and Criteria for Success

Performance Task -GRASPS Activity: Create a performance task using GRASPS GRASPS Performance Task Scenario (HO) Student roles and audiences (HO) Possible Products and Performances (HO)

Assessor Question 2: Determine achievement What specific characteristics in student responses, products, or performances should we examine to determine the extent to which the desired results were achieved? Criteria, rubrics, and exemplars are needed

Designing Scoring Rubrics Rubric: criterion-based scoring guide for evaluating a product or performance along a continuum. Consists of: Evaluative Criteria – qualities that must be met for work to measure up to a standard Fixed Measurement Scale – often 4 or 5 levels Indicators – descriptive terms for differentiating among degrees of understanding, proficiency, or quality

Rubric Types Holistic – provide an overall impression of the elements of quality and performance levels in a student’s work Analytic – divides a student’s performance into two or more distinct dimensions (criteria) and judges each separately Recommend use of analytic with a minimum of: Criteria for understanding (HO) Criteria for performance Using Facet-Related Criteria (Figure 8.3, Pg 178)

Rubric Types Generic – general criteria in given performance area Can be developed before specific task defined Example: General Problem Solving Rubric Example: Generic Rubric for Understanding (HO) Task-Specific – designed for use with particular assessment activity Task dependent so cannot be used to evaluate related performance tasks

Rubric Types Longitudinal Rubric – progression from naïve to sophisticated understanding Increased understanding of complex functions and interrelatedness of concepts Greater awareness of how discipline operates Greater personal control over and flexibility with knowledge

Effective Rubrics Relate specific task requirements to more general performance goals Discriminate among different degrees of understanding or proficiency according to significant features Do not combine independent criteria in one column of rubric Use Student Anchors to (Anchor design, Pg 181) Set standards based on student artifacts Consistency in judgment of student work Equip students to do more accurate and productive self- assessment

Effective Rubrics All potential performances should fit somewhere in rubric Rely on descriptive language (what quality looks like) not comparative or value language to make distinctions Avoid making lowest score point sound bad, should describe novice or ineffective performance Highlight judging performance’s impact as opposed to over rewarding just process or effort

Assessor Question 3: Valid and Reliable Does the proposed evidence enable us to infer a student’s knowledge, skill, or understanding? Validity: did we measure what we meant to measure Does the evidence indicate understanding of the expressed outcomes? Are the performances appropriate to the understanding sought? Do not pay so much attention to correctness that degree of understanding is lost.

Validity Two key validity questions for assessment tasks: A student could do well on this performance task, but really not demonstrate the understanding you are after? A student could perform poorly on this task, but still have significant understanding of the ideas and show them in other ways? Activity: determining validity (Figure 8.5)

Validity Two key validity questions for rubric: Could the proposed criteria be met but the performer still not demonstrate deep understanding? Could the proposed criteria not be met but the performer nonetheless still show understanding?

Reliability Reliable assessments reveal a credible pattern, a clear trend Need for multiple evidence (scrapbook) rather than just a snapshot of student performance Have parallel assessments on the same concept using multiple assessment formats.

Dr. Robert Mayes University of Wyoming Science and Mathematics Teaching Center