1. Assessment Literacy Series
-Orientation Module-
IMT Orientation Draft 02Sept11-CS

2. What is Assessment Literacy?
A process of designing, building, and reviewing performance measures for use in evaluating student achievement.

3. Series Objectives
Develop high-quality performance measures for use within the greater educator effectiveness system.
Increase assessment literacy of participants.
Gain a deeper understanding of academic content standards.

4. Assessment Life Cycle Examine Validity Evidence
Establish Assessment Purpose(s) and Design
Build Test Specifications & Blueprint
Develop Items
Develop Scoring Keys and Rubrics
Create Operational Forms & Administrative Guidelines
Review Forms
Administer Test and Report Results
Examine Validity Evidence

5. Overview of the Development Process
The Assessment Literacy training contains six (6) modularized components within three (3) action strands:
DESIGN
M1-Design & Purpose Statement
M2-Test Specifications & Blueprints
BUILD
M3-Item Specifications
M4-Scoring Keys & Rubrics
M5-Operational Forms & Administrative Guidelines
REVIEW
M6-Quality Assurance & Form Reviews

6. Development Process Purpose Statement Targeted Standards
Explain why measures are made, what they will measure, and how the results will be used.
Targeted Standards
Select content standards associated with the performance measure.

7. Development Process (cont.)
Test Specifications and Blueprints
Outline the types of items being used, point values, and depth of knowledge (DoK) distribution.
Guide item development by focusing items/tasks on discrete aspects of the content standards.
Item/Tasks Specifications
Articulate the type and characteristics of items/tasks that align to the test specifications and blueprints.

8. Development Process (cont.)
Scoring Keys & Rubrics
Provide answers for multiple choice items and detailed scoring rubrics for constructed response tasks.
Operational Forms
Organize and sequence items/tasks within an outlined format that includes item tags (to uniquely identify each item/task).

9. Development Process (cont.)
Administrative Guidelines
Provide detailed instructions for how to prepare, administer, and collect results.
Quality Assurance & Form Reviews
Review all operational forms, scoring keys/rubrics, and guidelines to ensure pre- established quality expectations have been met.
Determine the overall rigor and alignment at the item/task-level and for the overall operational form.

10. Helpful Materials Guides Handout #1 – Purpose Statement Examples
Handout #2 – Targeted Standards Example
Handout #3 – Test Specifications & Blueprint Example
Handout #4 – Depth of Knowledge (DoK) Chart
Handout #5 – Item Examples
Handout #6 – Scoring Key Example
Handout #7 – Rubric Examples
Handout #8 – Operational Form Example
Handout #9 – Administrative Guidelines Example
Handout #10 – Quality Assurance Checklist

11. Helpful Materials (cont.)
Templates
Template #1-Purpose Statement
Template #2-Targeted Standards
Template #3-Test Specifications & Blueprint
Template #4-Scoring Key
Template #5-Operational Form
Template #6-Administrative Guidelines

12. Helpful Material (cont.)
Stuff
“Smart Book”
Provides techniques and technical criteria for developing measures.
Performance Rigor Checklist
A quick screening tool used to examine item/task quality.

13. Principles of Well-developed Measures
Measures must:
Be built to achieve the designed purpose
Produce results that are used for the intended purpose
Align to targeted content standards
Contain a balance between depth and breadth of targeted content
Be standardized, rigorous, and fair
Be sensitive to testing time and objectivity
Be valid and reliable

14. Quality Assurance Rubrics
Evaluating Measures
Quality Assurance Rubrics
Three (3) versions-
Teacher
District
State or Vendor
Comprehensive tools that evaluate the technical process used in developing performance measures.
Organized into strands and provides an overall rating.

15. Decision-Making Approaches
Build Consensus Approach
All opinions expressed
Ideas of all participants gathered and synthesized
Group agreement reached
Team Leader Approach
Guiding questions
Group discussion about the pros and cons
Create a “best fit” course of action

16. Think-Pair-Share
5 minutes to think about the statement below and write down your thoughts
5 minutes to share your ideas with a partner
10 minutes to share with your group
“The Assessment Literacy process will help me as an educator by _______________.”

17. Summary & Next Steps Summary Orientation Module Next Steps
Introduced the Assessment Literacy process used to create high quality measures of student achievement.
Next Steps
Module 1: Design & Purpose Statement
Designing performance measures, including creating purpose statements and selecting academic content standards.