Understanding Depth of Knowledge North Middle/High School (OCSD5)

Revised Bloom’s Taxonomy Remember– Retrieve knowledge from long-term memory, recognize, recall, locate, identify Understand – Construct meaning, clarify, paraphrase, represent, translate, illustrate, give examples, classify, categorize, summarize, generalize, predict Apply – Carry out or use a procedure in a given situation; carry out or use/apply to an unfamiliar task Analyze – Break into constituent parts, determine how parts relate Evaluate – Make judgments based on criteria, check, detect inconsistencies/fallacies, critique Create – Put elements together to form a coherent whole, reorganize elements into new patterns/structures. We are all familiar with Bloom’s Taxonomy. It has long been a standard in educational circles and a key component in professional development. But, as a part of the shift to Common Core, there has also been a shift toward a Depth of Knowledge standard, which will be the standards used in creating Common Core Assessments. It is important that we thoroughly understand this shift in perspective and how it impacts our instructional purposes.

Depth of Knowledge (DOK) Adapted from the model used by Norm Webb, University of Wisconsin, to align standards with assessments Used by the Council of Chief State School Officers (CCSSO) for assessment alignment

Why Depth of Knowledge? No Child Left Behind (NCLB) requires assessments to “measure the depth and breadth of the state academic content standards for a given grade level” (U.S. Department of Education, 2003, p. 12)

What is Depth of Knowledge? The degree of depth or complexity of knowledge reflected in the content standards and assessments How deeply a student needs to understand the content for a given response/assessment It is essential all educators recognize the Depth of Knowledge in the revised content standards and objectives (CSOs). We need to understand the DOK of the CSOs to comprehend what we want kids to know, understand, and be able to do. If teachers don’t understand the DOK of the CSOs, they won’t be able to effectively instruct or assess. They won’t be able to prepare students for the DOK required on future assessments.

Webb’s DOK Levels Level 1 Recall and Reproduction Level 2 Skills & Concepts Level 3 Strategic Thinking Level 4 Extended Thinking

Depth of Knowledge Levels Level 1: Recall Recall, recognition; skill, behavior or sequence of behaviors learned through practice and easily performed Level 2: Skill/Concept Engagement of some mental processing beyond recalling; the use of information or conceptual knowledge; requires making some decisions regarding how to approach a question or problem Level 3: Strategic Thinking More sophisticated reasoning and analysis; deep understanding; students are required to solve problems & draw conclusions Level 4: Extended Thinking Requires integration of knowledge from multiple sources and the ability to represent knowledge in a variety of ways; it usually requires work over an extended period of time The DOK has 4 levels. Depth of Knowledge, or DOK, is a way to think about content complexity, not content difficulty. That is the first thing people must come to understand. Complexity is different from difficulty. For example, if students have not seen the word or content before, it might be difficult for them, but it is not complex. Level 1 DOK is recall and recognition. Level 2 DOK is about using a skill or a concept, i.e. Paraphrase. Conceptual understanding generally refers to the integration and application of concepts and other ideas within a content area. Procedural understanding denotes knowledge about skills and the sequence of steps, when and how these should be used appropriately, and their efficient and accurate applications. Level 3 DOK requires strategic thinking. Analysis and other examples are given here. Non-routine problem solving (e.g., reading and determining author’s purpose) is Level 3. Level 4 DOK requires extended thinking and usually requires work over a period of time, including gathering information, analyzing findings, preparing reports, and presenting findings. The new rigor of the Smarter Balanced Assessments will mean they contain more level 2 & 3 items.

DOK Level 1 Examples List animals that survive by eating other animals. Locate or recall facts explicitly found in text Describe physical features of places Determine the perimeter or area of rectangles given a drawing or labels Identify elements of music using musical terminology Identify basic rules for participating in simple games and activities Recall and reproduction Level 1 DOK 1 requires recall of information, such as facts, definitions, or performance of a simple process or procedure Answering a Level 1 item can involve following a simple, well-known procedure or formula. Simple skills and abilities or recall characterize DOK 1.

DOK Level 2 Examples Compare desert and tropical environments Identify and summarize the major events, problem, solution, conflicts in literary text Explain the cause and effect of historical events Predict a logical outcome based on information in a reading selection Explain how good work habits are important at home, school, and on the job. Classify plane and three dimensional figures Describe various styles of music DOK 2 includes the engagement of some mental processing beyond recalling or reproducing a response. Items require students to make some decisions as to how to approach the question or problem. These actions imply more than one mental or cognitive process/step.

DOK Level 3 Examples Compare consumer actions and analyze how these actions impact the environment Analyze or evaluate the effectiveness of literary elements (e.g. characterization, setting, point of view, conflict and resolution, plot structures) Solve a multiple-step problem and provide support with a mathematical explanation that justifies the answer Level 3 Strategic thinking DOK 3 requires deep understanding as exhibited through planning, using evidence, and more demanding cognitive reasoning. The cognitive demands at Level 3 are complex and abstract. An assessment item that has more than one possible answer and requires students to justify the response they give would most likely be a Level 3.

DOK Level 3 Examples Develop a scientific model for a complex idea Propose and evaluate solutions for an economic problem Explain, generalize or connect ideas, using supporting evidence from a text or source Create a dance that represents the characteristics of a culture

Understanding Depth of Knowledge DOK is about intended outcome, not difficulty. DOK is a reference to the complexity of mental processing that must occur to answer a question, perform a task, or generate a product. We must understand that DOK is not about the difficulty of the objective, rather, it is about the intended outcome or the complexity of the mental processing that must occur in order to get to the intended outcome. Consider the example of adding 4 + 4. This is easy. Students can do this by rote. It is a DOK 1. Increasing the difficulty of the problem by asking a student to add 4,678 + 7,530 does not increase the complexity. To solve this problem requires a child to recall the sequence of steps. The level of cognitive difficulty is still a DOK 1. It is still recall. Knowing the rules of adding larger numbers - increasing the difficulty of the addition problem - does not affect the intended outcome. We still want the student to be able to add, just larger numbers.

Understanding Depth of Knowledge Difficulty is a reference to how many students answer a question correctly: How many of you know the definition of exaggerate? DOK 1 - recall If all of you know the definition, this question is an easy question. How many of you know the definition of prescient? If most of you do not know the definition, this question is a difficult question. Copyright © 2007 Mississippi Department of Education Let’s look at another example of how the level of difficulty can be increased without increasing the DOK level. Do you know the definition of exaggerate? Do you know the definition of prescient? A common confusion is that in order to make an objective more difficult, we must increase the DOK level. This is simply NOT the case. The questions are asking the student to recall information…one definition might be considered less likely to be known than the other, but the objective remains the same…provide a definition. (RECALL). We can raise the level of difficulty WITHOUT raising the DOK level of the objective. Just in case you did not know, prescient (press-ee-ant, preshent, or pree-see-ent) is perceiving the significance of events before they occur or to know beforehand.

Understanding Depth of Knowledge Words like explain or analyze have to be considered in context. “Explain to me where you live” does not raise the DOK of a simple rote response. Even if the student has to use addresses or landmarks, the student is doing nothing more than recalling and reciting. Copyright © 2007 Mississippi Department of Education Even though the verbs (or action words) are listed on graphics explaining DOK, it is not the verb that is critically important, rather it is the words that follow the action. What is the objective really asking the student to do? In the above example, the word “Explain” may be perceived as a higher level objective, however, it simply asks the student to recall information.

Here is a quick reference wheel to help you identify the types of actions that are required for each DOK level.

Is DOK different from Bloom’s Taxonomy? The Depth of Knowledge is NOT determined by the verb, but the context in which the verb is used and the depth of thinking required.

DOK is about depth & complexity – not difficulty! DOK 3-Describe a model that you might use to represent the relationships that exist within the rock cycle. (Requires deep understanding of the rock cycle and a determination of how best to represent it) DOK 2-Describe the difference between metamorphic and igneous rocks. (Requires cognitive processing to determine the differences in the two rock types) DOK 1-Describe three characteristics of metamorphic rocks. (Simple recall) Same verb—used at all three DOK levels

Caution! The Depth of Knowledge is NOT determined by the verb, but the context in which the verb is used and the depth of thinking required.

To Summarize… Depth of Knowledge (DOK) is a scale of cognitive demand. DOK requires looking at the assessment item/standard in order to determine the level. DOK is about the item/standard-not the students’ ability. The context of the assessment item/standard must be considered to determine the DOK-not just a look at what verb was chosen.

Hess’ Cognitive Rigor Matrix: Depth + Bloom’s Level 1 Recall & Reproduction Level 2 Basic Skills & Concepts Level 3 Strategic Thinking & Reasoning Level 4 Extended Thinking Remember Recall, locate basic facts, details, events Understand Select appropriate words to use when intended meaning is clearly evident Specify, explain relationships, summarize, identify main ideas Explain, generalize, or connect ideas using supporting evidence Explain how concepts or ideas specifically relate to other content domains or concepts Apply Use language structure or word relationships to determine meaning Use contest to identify meaning of word, obtain and interpret information using text features Use concepts to solve non-routine problems Devise an approach among many alternatives to research a novel problem Analyze Identify whether information is contained in a graph, table, etc. Compare literacy elements, terms, facts, events, analyze format, organization, & text structures Analyze or interpret author’s craft to critique a text Analyze multiple sources or texts Analyze complex/ abstract themes Evaluate Cite evidence and develop a logical argument for conjectures Evaluate relevancy, accuracy, & completeness of information Create Brainstorm ideas about a topic Generate conjectures based on observations or prior knowledge Synthesis information within one source of text Synthesize information across multiple sources or texts For those of you who need a clearer connection between the Bloom’s Taxonomy of the past and the DOK levels of the future, Hess’ Cognitive Rigor Matrix may help you decipher the complex changes in perception that you need to undertake.

Questions, Comments, and Suggestions Please forward them to edward.murray@ocsd5.net See the examples in the next four slides (by subject area) and let me know if you cannot identify the appropriate DOK level.

Same VERB – three DOK Levels Science DOK? – Describe a model that you might use to represent the relationships that exist within the rock cycle DOK ? – Describe the differences between metamorphic and igneous rocks DOK ? – Describe three characteristics of metamorphic rocks DOK 3 (requires deep understanding of rock cycle and a determination of how best to represent it. DOK 2 (requires cognitive processing to determine the differences in the two types of rocks) DOK 1 (simple recall)

Same VERB – three DOK Levels Social Studies DOK ?  examine the amendment process of the West Virginia Constitution, give examples of amendments and explain why they occurred.  DOK ?   explain how nations benefit when they resolve conflicts peacefully.         DOK ? Use correct geographic terminology to explain direction, location, time zones, physical features of the earth. DOK 3 DOK 2 DOK 1

Same VERB – three DOK levels Secondary Mathematics DOK ? Explain what family of functions would best model a particular real-world situation. DOK ? Explain the difference between a linear and a quadratic function. DOK ? Explain how the “vertical line test is useful in determining whether or not a given graph is representing a function. DOK 3 DOK 2 DOK 1

Same VERB – three DOK levels Elementary Mathematics DOK ? Determine the formula for the area of a rectangle and explain reasoning through modeling DOK ? Demonstrate and model multiplication (repeated addition, arrays) and division (repeated subtraction, partitioning) DOK ? Model, describe, and draw – lines, rays, angles including right, obtuse, and acute angles. DOK 3 DOK 2 DOK 1