CTB CADDS Sally Valenzuela Director, Publishing Strategic Initiatives CTB/McGraw-Hill

2 Abstract  Evidence-based item development approaches require item authors to make explicit how items provide validity evidence to support claims.  Advances in technology require increased consideration of accessibility and interoperability during item authoring.  Refined item development processes are needed to meet these requirements.

3 CADDS  Serves as a bridge between standards and assessment  Provides opportunities for systematic unpacking of standards for item development  Documents steps in reaching assessment goals  Maintains focus at all steps on evidence to support interpretations and uses of test information

4 CADDS  Details information about the content to be assessed  Describes item types that will be used  Uses detailed specifications for each item/task  Increases direction for item writers

5 CADDS 1. Define the intended inferences and decisions to be based on test scores. 2. Define the achievement construct. 3. Draft performance level descriptors (expectations of students). 4. Define the evidence to be elicited by the item pool. 5. Complete item writer assignments to meet the item pool specification requirements. 6. Complete item creation (authoring and editing). 7. Field test assessment items and tasks. 8. Implement the operational test.

6 INFERENCE about students (Step 1) Achievement Construct (Step 2) Assessment item/task creation (Step 6) Item Writer Assignments (Step 5) Performance Level Descriptors (Step 3) Item Pool Evidence (Step 4) Pilot, field, operational testing (Step 7/8) DESIGNDEVELOPMENT

7 CADDS 1. Define intended inferences and uses of the assessment data. Adoption of CCSS has created need for transition assessments. Test designs are incorporating new item types. Reporting requirements are changing. “Alignment” issues abound.

8 CADDS 2. Define the test construct(s) that will become assessment targets. What cognitive tasks are required by the standard? How do we consider students progression of learning? What is the instructional context for a given standard? How do we incorporate performance levels?

9 CADDS 3. Develop initial proficiency level descriptors to guide development and interpretation of test scores. Identify source(s) of PLDS. Define the role of PLDs in item specifications and authoring.

10 CADDS 4. Define the evidence. 1. Item pool or test blueprint 2. Specifications 3. Item/task templates 4. Instructions to item writers 5. Develop items and performance tasks based on specifications. 1. specifications templates 2. cognitive task frameworks 3. tagging for accessibility and interoperability

11 CADDS 6. Refine items and tasks through collaborative review by stakeholders. 7. Field test items and tasks in appropriate small- or large- scale settings. 8. Implement the operational test and continue the design and specification validation process.

12 Implementation  Expansion of item specifications template Performance descriptionRules for source materials Performance level descriptorsRules for item/task problem Grade level placementRules for response requirements Assessment targets/standardsAdministration requirements 21 st century skillsAccessibility requirements Cognitive rigorAdministrator directions Problem/processing typeTechnology requirements Performance descriptionRules for source materials

13 Implementation  Articulation of cognitive tasks during item development  Cognitive Task Frameworks Traditional (DOK, Bloom’s) Within CCSS (Conley) – Mathematical Practice Standards – Selected ELA standards Cognitive Rigor Matrix (Hess)

14 Item Authoring  Clearly defined elements for authors construct evidence requirements parameters for student responses options for item type, cognitive demand, other variable factors  Articulated focus on evidence, validity and accessibility

15 CCSS Standard Modeling with Geometry G-MG Apply Geometric Concepts in Modeling Situations. Model with Mathematics Make sense of problems. Attend to precision. Item 1 Item 2 Item 3 Above Proficient Proficient Basic Item 1 Item 2 Item 3 Above Proficient Proficient Basic Item 1 Item 2 Item 3 Above Proficient Proficient Basic

16 Item Development Implications  Explicit articulation of evidence to support claims  Rigorous specifications development  More direction for item writers  Clearer distinction among item attributes  Refinement of item development plans