Assessment Lifecycle 1 Dr. Michele Walker, Director Office of Student Assessment

Collaborative Effort 2 IDOE VendorEducators

Item Specifications/ Test Blueprints Item Development Internal IDOE Item Reviews Revise/Select Items Bias/Sensitivity, Content Reviews Administer Pilot Items (Statistical sample) Create / Administer Operational Test Create / Administer Operationalized Field Test (Operational and Pilot combined ) The Journey—From Item Specifications to the Operational Test OR A 10- to 12-month process An 18- to 24-month process Blue font = educator involvement

Establishing Item Specifications: An Overview 4 START analyze standards to determine DOK analyze standards to determine “weight” assign item format (MC, GR, CR, ER, TE) develop “specifications” for standards develop “content limits” for standards

Depth of Knowledge (DOK) Levels Level 1: Recall – recall of a fact, information, or procedure Level 2: Skill/Concept – use information or conceptual knowledge, two or more steps, etc. Level 3: Strategic Thinking – requires reasoning, developing a plan or a sequence of steps, some complexity, more than one possible answer Level 4: Extended Thinking – requires making connections across content areas or within a content area; an investigation, time to think and process multiple conditions of the problem

Establishing Item Specifications… Assign DOKs at the standard level Reflect the complexity of the objective, rather than the difficulty Describes the thinking involved, not the likelihood that the task will be completed correctly Assign standards “weights” 3 = essential 2 = important 1 = introductory 0 = assessed in the classroom

Establishing Item Specs… (continued) Assign item formats Multiple-Choice (MC) Gridded-Response (GR) Constructed-Response (CR) Extended-Response (ER) Technology-Enhanced (TE) Develop specifications and limits Clarify intention of standard Describe appropriate ways to assess standard Identify appropriate language and vocabulary Establish any content limits Provide examples of appropriate content/contexts

What is a “specification”? 5.2 ComputationSpecification Computation- Students solve problems involving multiplication and division of whole numbers and solve problems involving addition, subtraction, and simple multiplication and division off fractions and decimals Solve problems involving multiplication and division of whole numbers. Use straight computation. May include remainders, but no remainders in context. IDOE Content Specialists and educators identify appropriate language and vocabulary for items. Additional information to inform item development

What is a “content limit”? 5.2 ComputationContent Limit Computation- Students solve problems involving multiplication and division of whole numbers and solve problems involving addition, subtraction, and simple multiplication and division off fractions and decimals Solve problems involving multiplication and division of whole numbers. Divide 4-digit by 2-digit (maximum). Multiply 2-digit by 3-digit (maximum). Item parameters

The Basic Test Blueprint PercentDerived Test Blueprint Summary Standard 1 Standard 2 Standard 3 Etc. 100% Total Test

Involving Indiana Educators: Item Specs/Limits/Blueprint Summary Analyzing standards to determine – DOK level (1, 2, 3, 4) – standard weight (3, 2, 1, 0) – appropriate item format (MC, GR, CR, ER, TE) – specifications – content limits – blueprint percentages

Content Review Indiana educators in grade/content committees verify that each item is… matched to an Indiana Academic Standard accurate and appropriate for grade level and difficulty range clearly stated and unambiguous appropriate for the assigned DOK

Bias/Sensitivity Review Test items must be accessible to a diverse student population with respect to gender, race, ethnicity, geographic region, disability, socioeconomic status, and other factors. Test items must not contain: Offensive, disturbing, or inappropriate language or content Stereotyping Insensitivity to historical representation of groups

Final Steps in the Process…. Revise items as a result of Content and Bias/Sensitivity Reviews Select items for Pilot and Operational test