Diving into Depth of Knowledge in an ELA classroom Amy Allison, Assessment Technology Specialist, ISD Kristin Edlund, Secondary ELA Specialist, ISD

Connection to Danielson Framework Criterion Domain/Area Criterion 1: Centering instruction on high expectations for student achievement. 3c. Instruction: Engaging Students in Learning Criterion 2: Demonstrating effective teaching practices. 3b. Instruction: Using Questioning And Discussion Techniques Criterion 4: Providing clear and intentional focus on subject matter content and curriculum. 1a. Planning and Preparation: Demonstrating Knowledge of Content and Pedagogy

Objectives Teachers will learn the four different depth of knowledge levels. Teachers will apply depth of knowledge to their teaching.

What is Depth of Knowledge? Depth of Knowledge is the cognitive expectation demanded of the students by the standards, curricular activities, and assessment tasks (Webb, 1997). Amy

Team Task - Sort With your team, Read each of the tasks. Sort into four categories. Depth of Knowledge Level 1 Depth of Knowledge Level 2 Depth of Knowledge Level 3 Depth of Knowledge Level 4 Kristin

Webb’s Model DOK Level Title of Level 1 Recall and Reproduction 2 Skills and Concepts 3 Short-term Strategic Thinking 4 Extended Thinking Amy Webb created a model that curricular and assessment elements may be categorized based on the cognitive demands required to produce an acceptable response.   Each grouping of tasks reflects a different level of cognitive expectation or depth of knowledge to complete the task.

DOK Reading NEW/KNEW/RE-NEW This is NEW to me! I KNEW that! That's a new way to think about it! Kristin Processing strategy for the reading Picture of the doc

How do I know what the DOK is? Reflect the level work students are most commonly required to perform By the complexity not the difficulty By the cognitive demand required of the student Amy  DOK level are assigned to each course objective the following served as general guidelines:   The DOK level assigned should reflect the level of work students are most commonly required to perform in order for the response to be deemed acceptable.   The DOK level should reflect the complexity of the cognitive processes demanded by the task outlined by the objective, rather than its difficulty. Ultimately the DOK level describes the kind of thinking required by a task, not whether or not the task is “difficult”.   If there is a question regarding which of two levels a statement addresses, such as Level 1 or Level 2, or Level 2 or Level 3, it is appropriate to select the higher of the two levels.   The DOK level should be assigned based upon the cognitive demands required by the central performance described in the objective.   The objective’s central verb(s) alone is/are not sufficient information to assign a DOK level. Developers must also consider the complexity of the task and/or information, conventional levels of prior knowledge for students at the grade level, and the mental processes used to satisfy the requirements set forth in the objective.

Posted by: Erik M. Francis Amy Difficulty refers to the amount of effort that the learner must expend within a level of complexity to accomplish a learning objective. It is possible for a learning activity to become increasingly difficult without becoming more complex. Difficult questions generally have one final, specific answer, outcome, solution, or end result that is either correct or incorrect.  Complexity describes the thought process that the brain uses to deal with information.  Complex questions, problems, or tasks may not have a simple, straightforward, or succinct answer, solution, or outcome.  Complex questions may have more than one possible response.  Complex problems may have more than one possible resolution.  Complex tasks may have more than one possible result. Complex questions, problems, and tasks also allow students to delve deeper into the content, concepts, ideas, subjects, and topics being taught and learned.  Instead of merely learning what things are and how things work, complex questions, problems, and tasks engage students to establish and examine relationships, explore causes and effects, and consider options and possibilities. Taken from: http://maverikeducation.blogspot.com/2014/03/difficulty-vs-complexity-whats.html Posted by: Erik M. Francis

Difficult vs. Complex Recite Hamlet's soliloquy from Act III, Scene 1 Use the word abnegation correctly in a sentence Explain what a preposition is and the role of prepositions or prepositional phrases in a text. Compare Hamlet's dilemma with Simba's in The Lion King Contrast the tones created by the use of the words steal and abscond in the two paragraphs. Read the two passages, including one with the prepositional phrases removed. How is the text affected?  Find a place in your writing to improve by adding a prepositional phrase. Kristin Have teacher talk about the difference between the two

Web Alignment Tool by Norman L. Webb, 2005 Include as handout Amy Kristin(example: cite evidence) ƒ The task’s or objective’s central verb(s) alone is/are not sufficient to assign a DOK level. Developers must consider “what comes after verb” - the complexity of the task and content/concepts -- in addition to the mental processing required by the requirements set forth in the objective (Webb’s Depth of Kno https://education.ohio.gov/getattachment/Topics/Teaching/Educator-Evaluation-System/How-to-Design-and-Select-Quality-Assessments/Webbs-DOK-Flip-Chart.pdf.aspxwledge Guide). Web Alignment Tool by Norman L. Webb, 2005

Level 1: Recall & Reproduction Work with facts, terms, details, principles, properties Student knows the answer or not Limited cognitive demand Amy Kristin (example) Curricular elements that fall into this category involve basic tasks that require students to recall or reproduce knowledge and/or skills. The subject matter content at this level usually involves working with facts, terms, details, calculations, principles, and/or properties. It may also involve use of simple procedures or formulas. There is little or no transformation of the target knowledge or skill required by the tasks that fall into this category. A student answering a Level l item either knows the answer or does not; that is, the answer does not need to be figured out”

Level 1: Recall & Reproduction Teacher Role Student Role Questions to direct or focus attention Models Provides examples Examines Leads Breaks down Defines Recognizes Responds Remembers Memorizes Restates Absorbs Describes Demonstrates Follows directions Applies routine processes, definitions, and procedures Amy – Kristin What do you notice about the teacher and student roles? (Who is doing the work) Danielson https://education.ohio.gov/getattachment/Topics/Teaching/Educator-Evaluation-System/How-to-Design-and-Select-Quality-Assessments/Webbs-DOK-Flip-Chart.pdf.aspx Kristin:  pare down, bullet list

Level 1: Recall & Reproduction Products Fill in the blank tasks Edit sentences Identify/write sentence types Highlight key words Follow directions Show and tell Locate or recall quotes Document/cite sources Brainstorm Identify parts of speech Label or locate parts in a diagram Vocabulary definitions Decoding words Level 1 tasks let us check for background knowledge or understanding

With your partner, create one or two examples of a DOK level 1 task. Partner Task With your partner, create one or two examples of a DOK level 1 task. Kristin (Picture) Level 1 tasks are still important – they show students' understanding.  What's a level 1 task you use(or might use)in your class?

Level 1: Recall & Reproduction Examples Describe an event, character, setting, etc. in a story Write a list of key words you know about... Recite/recall a fact or date related to ... Write/retell in your own words ... Cut out, draw, or match a picture that illustrates an event, process, or story Retell in your own words/paraphrase Use a dictionary, glossary, or thesaurus to find word meanings Locate or retrieve information in verbatim form to answer a question Edit applying a standard set of conventions and/or criteria that should eventually be automated (e.g., applying rules for grammar)

Level 2: Skill/Concept Compare or differentiate among people, places, events, objects, text types Apply multiple concepts when responding Classify items into meaningful categories Explain relationships Provide and explain examples and non-examples Amy  Level 2 includes the engagement of mental processing beyond recalling, reproducing, or locating an answer. This level generally requires students to compare or differentiate among people, places, events, objects, text types, etc.; apply multiple concepts when responding; classify or sort items into meaningful categories; describe or explain relationships, such as cause and effect, character relationships; and provide and explain examples and non-examples. A Level 2 “describe or explain” task requires students to go beyond a basic description or definition to predict a possible result or explain “why” something might happen. The learner makes use of information provided in context to determine intended word meanings, which tools or approach is appropriate to find a solution (e.g., in a math word problem), or what characteristics to pay attention to when making observations. At this level, students are asked to transform/process target knowledge before responding. Example mental processes that often denote this particular level include: summarize, estimate, organize, classify, extend, and make basic inferences.  https://education.ohio.gov/getattachment/Topics/Teaching/Educator-Evaluation-System/How-to-Design-and-Select-Quality-Assessments/Webbs-DOK-Flip-Chart.pdf.aspx

Level 2: Skill/Concept Teacher Role Student Role Questions to differentiate, infer, or check conceptual understanding Models Organizes/ reorganizes Explores possible options or connections Provides examples and non-examples  Solves routine problems/tasks involving multiple decision points and concepts Constructs models to show relationships Demonstrates use of conceptual knowledge Compiles and organizes Illustrates/explains with examples or models Examines https://education.ohio.gov/getattachment/Topics/Teaching/Educator-Evaluation-System/How-to-Design-and-Select-Quality-Assessments/Webbs-DOK-Flip-Chart.pdf.aspx

Level 2: Skill/Concept Possible Products Captioned Photos Summary Timeline Demonstration Presentation Interview Journal Entry Graphic Organizer Relationship Mind Maps Blog Commenting Survey Development https://education.ohio.gov/getattachment/Topics/Teaching/Educator-Evaluation-System/How-to-Design-and-Select-Quality-Assessments/Webbs-DOK-Flip-Chart.pdf.aspx

With your partner, create one or two examples of a DOK level 2 task. Partner Task With your partner, create one or two examples of a DOK level 2 task.

Level 2: Skill/Concept Examples Sequence a key chain of events and supporting details using a timeline, cartoon strip, outline or flow chart Write a summary /informational report or develop an outline of central ideas and supporting details Make a diorama to illustrate/explain an event Create a questionnaire or survey to answer a question Participate in a simulation in order to understand and describe differing perspectives Write a diary/blog entry for a character or historical figure

Level 3: Strategic Thinking and Reasoning Analysis Evaluation Explore Questions with Multiple Outcomes Provide Evidence Integration of conceptual knowledge and multiple skills Focus on one item (i.e. text, investigation, key source) Tasks and classroom discourse falling into this category demand the use of planning, reasoning, and higher order thinking processes, such as analysis and evaluation, to solve real-world problems or explore questions with multiple possible outcomes. Stating one’s reasoning and providing relevant supporting evidence are key markers of DOK 3 tasks. The expectation established for tasks at this level require an in-depth integration of conceptual knowledge and multiple skills to reach a solution or produce a final product. DOK 3 tasks and classroom discourse focus on in-depth understanding of one text, one data set, one investigation, or one key source, whereas DOK 4 tasks expand the breadth of the task using multiple texts or sources, or multiple concepts/disciplines to reach a solution or create a final product. https://education.ohio.gov/getattachment/Topics/Teaching/Educator-Evaluation-System/How-to-Design-and-Select-Quality-Assessments/Webbs-DOK-Flip-Chart.pdf.aspx

Level 3: Strategic Thinking and Reasoning Teacher Role Student Role Questions to probe reasoning and underlying thinking Asks open-ended questions Provides criteria and examples Encourages multiple approaches and interpretations  Determines when/where (text, concept) depth and exploration is most appropriate  Uncovers and selects  supporting evidence Critiques Analyzes Debates Makes claims and judgments Plans Initiates questions Argues Sustains inquiry into topics or deeper problems Applies to the real world  https://education.ohio.gov/getattachment/Topics/Teaching/Educator-Evaluation-System/How-to-Design-and-Select-Quality-Assessments/Webbs-DOK-Flip-Chart.pdf.aspx

Level 3: Strategic Reasoning Possible Products Debate from a given perspective Multi-paragraph essay or short story Literary critique Play, book, music, or movie review Informational report with several subtopics Fact-based argument (Is this criticism supported by the historical facts?) https://education.ohio.gov/getattachment/Topics/Teaching/Educator-Evaluation-System/How-to-Design-and-Select-Quality-Assessments/Webbs-DOK-Flip-Chart.pdf.aspx

Level 3: Strategic Reasoning Examples Write a letter to the editor after evaluating a product Use evidence to generate criteria for making judgments Make a booklet or brochure about a topic, organization, or issue Participate on a panel to discuss differing viewpoints on... Prepare a speech to support your perspective about ... Write an essay, short story, poem, or play Propose an alternate solution to a problem studied

Which one of these tasks is a level 3 task? How do you know? Product:  create a "trailer" video that includes evidence from the text: Task A: Use a central theme that is expressed across all three texts to show how authors use symbolism and metaphor in developing their theme. Task B: Summarize the story, including only the most important key events, characters, and settings involved in each stage of the story. Task C: Explain the central theme of a text, showing the interrelationships between the characters and events in the story.  Task C is the level 3 task! Which level are the other 2?  Why? Task A is level 4 –synthesize ideas across multiple texts,AND applies concepts taught Task B is level 2 – only uses one story,summarizes Level 3 & 2 are where we want to see most of our LA work in reading In writing, level 4!

Level 4: Extended Thinking Use of multiple texts/sources to reach a solution or create a final product Use of multiple concepts/disciplines to reach a solution or create a final product Sustain strategic thinking over a longer period of time Authentic problems and audiences Collaboration within a project-based setting Curricular elements assigned to this level demand extended and integrated use of higher order thinking processes such as critical and creative-productive thinking, reflection, and adjustment of plans over time. Students are engaged in conducting multi-faceted investigations to solve real-world problems with unpredictable solutions. Employing and sustaining strategic thinking processes over a longer period of time to solve the problem or produce an authentic product is a key feature of curricular objectives assigned to DOK 4. Key aspects that denote this particular level typically include authentic problems and audiences, and collaboration within a project-based setting. DOK 3 tasks and classroom discourse focus on in-depth understanding of one text, one data set, one investigation, or one key source, whereas DOK 4 tasks expand the breadth of the task using multiple texts or sources, or multiple concepts/disciplines to reach a solution or create a final product. https://education.ohio.gov/getattachment/Topics/Teaching/Educator-Evaluation-System/How-to-Design-and-Select-Quality-Assessments/Webbs-DOK-Flip-Chart.pdf.aspx

Level 4: Extended Thinking Teacher Role Student Role Questions to extend thinking and broaden perspectives Facilitates teaming, collaboration, self-evaluation Designs Takes risks Researches synthesizing multiple resources Collaborates Plans, organizes, and modifies Creates concrete tangible products https://education.ohio.gov/getattachment/Topics/Teaching/Educator-Evaluation-System/How-to-Design-and-Select-Quality-Assessments/Webbs-DOK-Flip-Chart.pdf.aspx

Level 4: Extended Thinking Possible Products Short film/documentary Agency presentation Research report Play Essay Newspaper or series of articles Multi-Media product Anthology of original writing, art, music, etc. https://education.ohio.gov/getattachment/Topics/Teaching/Educator-Evaluation-System/How-to-Design-and-Select-Quality-Assessments/Webbs-DOK-Flip-Chart.pdf.aspx

Level 4: Extended Thinking Activities Applying information from more than one discipline to solve open- ended problems in new or real-world situations Tasks that require multiple roles and collaboration and coordination with others (e.g., script writing, camera work, editing, and acting/ talent) Tasks that involve drawing evidence from multiple sources to support solutions/conclusions  Organizing/conducting a community service project or school-based event Writing original pieces that apply skills and concepts taught

Team Task – Sort Again With your team, Resort into four categories. Depth of Knowledge Level 1 Depth of Knowledge Level 2 Depth of Knowledge Level 3 Depth of Knowledge Level 4 Kristin Did anything change from the first sort to now?

Partner Activity Choose a task from the sort Level it up! Share with group Pick Level 1 or 2 task from a card – make it a level 3 or 4 Model an example

Scaffolding Level 3 and 4 Tasks Graphic Organizer (show examples – how graphic organizers scaffold complex thinking Theme pyramid)

Modify tasks from the curriculum and up the depth of knowledge Time to work! Modify tasks from the curriculum and up the depth of knowledge

Resources Hess, K. (2013). A Guide for Using Webb’s Depth of Knowledge with Common Core State Standards. Retrieved from https://education.ohio.gov/getattachment/Topics/Teaching/Educator- Evaluation-System/How-to-Design-and-Select-Quality- Assessments/Webbs-DOK-Flip-Chart.pdf.aspx

Contact us! Amy Allison, allisona@Issaquah.wednet.edu Kristin Edlund, edlundk@Issaquah.wednet.edu