Introduction to Depth of Knowledge Jamie Marantz Director Core Learning 2.28.13 Dublin Unified School District Presentation

Dublin Unified School District Presentation Learning Targets I will be able to evaluate assessment questions using Webb’s Depth of Knowledge Levels. 2.28.13 Dublin Unified School District Presentation

Why is Depth of Knowledge (DOK) Important? Use it to ensure that instructional and assessment activities match the cognitive rigor required to master the standards. 2.28.13 Dublin Unified School District Presentation

Dublin Unified School District Presentation DOK is NOT… … about the ‘TYPE’ of thinking Verbs (i.e. Bloom’s) such as analyzing, comprehending, recalling …about the ‘DIFFICULTY’ Difficulty is about how many students answer a question correctly. 2.28.13 Dublin Unified School District Presentation

Same Verb does Not Equal Same TYPE or DEPTH of Thinking While verbs may point to a DOK level, it is what comes after the verb that is the best indicator of the rigor/DOK level. DOK 1- Describe three characteristics of plant cells DOK 2- Describe the difference between plant and animal cells. DOK 3- Describe a model that you might use to represent a key relationship between plant and animal cells based on their functions. 2.28.13 Dublin Unified School District Presentation

DOK is not about difficulty... Difficulty refers to how many students answer a question correctly. DOK 1 – recall: “How many of you know what ‘angry’ means?” If you know, it’s an easy question. “Who knows what ‘prescient’ means?” If most do not know, then you have a ‘hard’ or difficult seeming question. 2.28.13 Dublin Unified School District Presentation

DOK is about COMPLEXITY DOK refers to the complexity of the mental processing students must perform to complete a task, answer a question or create a product. How deeply do you have to know the content to complete the task. Instruction and assessment activities must reflect the DOK level of the objective or intended learning outcome. 2.28.13 Dublin Unified School District Presentation

Dublin Unified School District Presentation DOK Levels DOK-1 : Recall, Remember and Reproduction DOK-2 : Basic Application of Skills & Concepts DOK-3 : Strategic Thinking DOK-4 : Extended Thinking 2.28.13 Dublin Unified School District Presentation

Level 1- Recall and Reproduction DOK 1 requires recall of information, such as a fact, definition, term, or performance of a simple process or procedure. You could memorize something simple, or complex DOK 1 can involve perform a routine procedure. 2.28.13 Dublin Unified School District Presentation

Dublin Unified School District Presentation Level 1 Examples Find facts explicitly stated in a text. Describe the components of a group. Identify elements from the periodic table from given facts. Identify basic rules for performing subtraction of double digit numbers. 2.28.13 Dublin Unified School District Presentation

Level 2 – Skills and Concepts DOK 2 requires Organizing of information in some way Using some mental processing like determining relationships between items beyond recalling Items can require students to make some decisions as to how to approach the question or problem. These actions imply more than one cognitive process/step. For DOK 1 + 2, there’s usually one ‘right’ answer. 2.28.13 Dublin Unified School District Presentation

Dublin Unified School District Presentation Level 2 - Examples Explain the cause-effect of a collision using Newton’s 2nd Law. Distinguish fact from opinion in a text. Compare tundra and arctic seasons. Identify and summarize the major events, problem, solution, conflicts in a literary text. 2.28.13 Dublin Unified School District Presentation

Level 3 – Strategic Thinking DOK 3 questions can have multiple ways to get to the right answer or multiple right answers. DOK 3 requires deep understanding as exhibited through reasoning, planning, and more demanding cognitive reasoning. The cognitive demands are often complex and/or abstract. DOK 3 questions often have more than one possible answer and often require students to justify the response they give using and explaining evidence for their reasoning. 2.28.13 Dublin Unified School District Presentation

Dublin Unified School District Presentation Level 3 - Examples Create a children’s story that represents the characteristics of a culture Develop a scientific model for the results of our experiment Explain, the most significant effect of WWII on the nations of Europe Propose and evaluate solutions for a social problem 2.28.13 Dublin Unified School District Presentation

Level 4 – Extended Thinking DOK 4 requires high cognitive demand and is very complex. Students are expected to make connections—relate ideas within the content or among content areas—and have to select or devise one approach among many alternatives on how the situation can be solved. Due to the complexity of cognitive demand, DOK 4 often requires an extended period of time. 2.28.13 Dublin Unified School District Presentation

Dublin Unified School District Presentation Level 4 – Examples Gather, analyze, organize, and interpret information from multiple (print and non print sources) to draft a reasoned report. Write a report that illustrates how multiple themes (historical, geographical, social) may be interrelated. Develop generalizations of the results obtained or strategies we used and apply them to this new problem. 2.28.13 Dublin Unified School District Presentation

DOK levels can be cumulative An item/standard written to DOK 3 often contains DOK 1 and DOK 2 level demands. 2.28.13 Dublin Unified School District Presentation

Assigning Depth of Knowledge to Assessments Activity Read each of the following assessment Items and Assign a Depth of Knowledge level to each Question and then compare with others. Justify your reasoning. Use language structures (Pre/suffix) or word relationships (antonym/synonym) to determine word meanings. Analyze/interpret author’s craft (literary devices, viewpoint, potential bias) to critique the text. Explain the relationship between the action and resolution of the story. 2.28.13 Dublin Unified School District Presentation

Transitioning to Next Generation Assessments Ingrid Roberson Professional Expert Assessment

Dublin Unified School District Presentation Learning Targets I will be able to assess the Depth of Knowledge required of individual questions. I will be able to identify key differences between the California Standards Test (CST) and SBAC. 2.28.13 Dublin Unified School District Presentation

CST/SBAC Compare and Contrast Activity Task: Compare and contrast the California Standards Test and the Smarter Balanced Assessment. 2.28.13 Dublin Unified School District Presentation

CST/SBAC Compare and Contrast Activity CST Review (20 minutes) As a pair, select 4 CST questions across different strands. As a pair, review the first CST question together, assigning a Depth of Knowledge level and citing evidence for your answer. Individually review the 3 remaining CST questions. Now, share your results and reasoning with your partner. http://starsamplequestions.org 2.28.13 Dublin Unified School District Presentation

CST/SBAC Compare and Contrast Activity SBAC Review (20 minutes) As a pair, select 4 SBAC questions across different strands. As a pair, review the first SBAC question together, assigning a Depth of Knowledge level and citing evidence for your answer. Individually review the 3 remaining SBAC questions. Now, share your results and reasoning with your partner. http://www.smarterbalanced.org 2.28.13 Dublin Unified School District Presentation

CST/SBAC Compare and Contrast Activity Conclusion (10 minutes) As a pair, decide what is the most important difference between the CST and the SBAC? Discuss the effects of this difference for your instruction. 2.28.13 Dublin Unified School District Presentation

Unpacking the Smarter Balanced Assessments Ingrid Roberson Professional Expert Assessment 2.28.13 Dublin Unified School District Presentation

Dublin Unified School District Presentation Learning Targets I can describe main features of the SMARTER Balanced Assessment Consortium (SBAC). TIME: 10 minutes (CONTINUED for Slides 58 to 63) INTENT: To provide an overview of the broad features of SMARTER Balanced Assessment Consortium’s (SBAC’s) assessment plan __________________________________________________________________________________ _ There are still many unanswered questions about what the 2014 assessments will look like. Here is what we know: Although originally associated with PAARC, California has recently signed on with SBAC and has opted to take a governing role. This means California educators will take a strong role in developing the assessments. Unlike current CST assessments, the new CCSS assessments will be computer-based and be adaptive. Adaptive means that the questions will adjust in difficulty level as the student answers. This will prevent frustration and burnout as students take the test—and thus give a more accurate read of their proficiency levels. In addition to end of the year summative assessment, SBAC will provide item banks and performance assessments for districts to locally administer benchmarks. Performance assessments will be both single period and extended—over several days and incorporating speaking and listening tasks. 2.28.13 Dublin Unified School District Presentation

Ingrid Roberson, Professional Expert Assessment iroberson@acoe.org Thank you! Ingrid Roberson, Professional Expert Assessment iroberson@acoe.org Jamie Marantz, Director, Core Learning jmarantz@acoe.org 2.28.13 Dublin Unified School District Presentation