1. Diving into Depth of Knowledge in an ELA classroom
Amy Allison, Assessment Technology Specialist, ISD
Kristin Edlund, Secondary ELA Specialist, ISD

2. Connection to Danielson Framework
Criterion
Domain/Area
Criterion 1: Centering instruction on high expectations for student achievement.
3c. Instruction: Engaging Students in Learning
Criterion 2: Demonstrating effective teaching practices.
3b. Instruction: Using Questioning And Discussion Techniques
Criterion 4: Providing clear and intentional focus on subject matter content and curriculum.
1a. Planning and Preparation: Demonstrating Knowledge of Content and Pedagogy

3. Objectives
Teachers will learn the four different depth of knowledge levels.
Teachers will apply depth of knowledge to their teaching.

4. What is Depth of Knowledge?
Depth of Knowledge is the cognitive expectation demanded of the students by the standards, curricular activities, and assessment tasks (Webb, 1997).
Amy

5. Team Task - Sort With your team, Read each of the tasks.
Sort into four categories.
Depth of Knowledge Level 1
Depth of Knowledge Level 2
Depth of Knowledge Level 3
Depth of Knowledge Level 4
Kristin

6. Webb’s Model DOK Level Title of Level 1 Recall and Reproduction 2
Skills and Concepts 3
Short-term Strategic Thinking 4
Extended Thinking
Amy
Webb created a model that curricular and assessment elements may be categorized based on the cognitive demands required to produce an acceptable response.  
Each grouping of tasks reflects a different level of cognitive expectation or depth of knowledge to complete the task.

7. DOK Reading NEW/KNEW/RE-NEW This is NEW to me! I KNEW that!
That's a new way to think about it!
Kristin
Processing strategy for the reading
Picture of the doc

8. How do I know what the DOK is?
Reflect the level work students are most commonly required to perform
By the complexity not the difficulty
By the cognitive demand required of the student
Amy 
DOK level are assigned to each course objective the following served as general guidelines: 
 The DOK level assigned should reflect the level of work students are most commonly required to perform in order for the response to be deemed acceptable. 
 The DOK level should reflect the complexity of the cognitive processes demanded by the task outlined by the objective, rather than its difficulty. Ultimately the DOK level describes the kind of thinking required by a task, not whether or not the task is “difficult”. 
 If there is a question regarding which of two levels a statement addresses, such as Level 1 or Level 2, or Level 2 or Level 3, it is appropriate to select the higher of the two levels. 
 The DOK level should be assigned based upon the cognitive demands required by the central performance described in the objective. 
 The objective’s central verb(s) alone is/are not sufficient information to assign a DOK level. Developers must also consider the complexity of the task and/or information, conventional levels of prior knowledge for students at the grade level, and the mental processes used to satisfy the requirements set forth in the objective.

9. Posted by: Erik M. Francis
Amy
Difficulty refers to the amount of effort that the learner must expend within a level of complexity to accomplish a learning objective. It is possible for a learning activity to become increasingly difficult without becoming more complex. Difficult questions generally have one final, specific answer, outcome, solution, or end result that is either correct or incorrect. 
Complexity describes the thought process that the brain uses to deal with information.  Complex questions, problems, or tasks may not have a simple, straightforward, or succinct answer, solution, or outcome.  Complex questions may have more than one possible response.  Complex problems may have more than one possible resolution.  Complex tasks may have more than one possible result.
Complex questions, problems, and tasks also allow students to delve deeper into the content, concepts, ideas, subjects, and topics being taught and learned.  Instead of merely learning what things are and how things work, complex questions, problems, and tasks engage students to establish and examine relationships, explore causes and effects, and consider options and possibilities.
Taken from:
Posted by: Erik M. Francis

10. Difficult vs. Complex Recite Hamlet's soliloquy from Act III, Scene 1
Use the word abnegation correctly in a sentence
Explain what a preposition is and the role of prepositions or prepositional phrases in a text.
Compare Hamlet's dilemma with Simba's in The Lion King
Contrast the tones created by the use of the words steal and abscond in the two paragraphs.
Read the two passages, including one with the prepositional phrases removed. How is the text affected?  Find a place in your writing to improve by adding a prepositional phrase.
Kristin
Have teacher talk about the difference between the two

11. Web Alignment Tool by Norman L. Webb, 2005
Include as handout
Amy
Kristin(example: cite evidence)
ƒ The task’s or objective’s central verb(s) alone is/are not sufficient to assign a DOK level. Developers must consider “what comes after verb” - the complexity of the task and content/concepts -- in addition to the mental processing required by the requirements set forth in the objective (Webb’s Depth of Kno
Guide).
Web Alignment Tool by Norman L. Webb, 2005

12. Level 1: Recall & Reproduction
Work with facts, terms, details, principles, properties
Student knows the answer or not
Limited cognitive demand
Amy
Kristin (example)
Curricular elements that fall into this category involve basic tasks that require students to recall or reproduce knowledge and/or skills. The subject matter content at this level usually involves working with facts, terms, details, calculations, principles, and/or properties. It may also involve use of simple procedures or formulas. There is little or no transformation of the target knowledge or skill required by the tasks that fall into this category. A student answering a Level l item either knows the answer or does not; that is, the answer does not need to be figured out”

13. Level 1: Recall & Reproduction
Teacher Role
Student Role
Questions to direct or focus attention
Models
Provides examples
Examines
Leads
Breaks down
Defines
Recognizes
Responds
Remembers
Memorizes
Restates
Absorbs
Describes
Demonstrates
Follows directions
Applies routine processes, definitions, and procedures
Amy – Kristin
What do you notice about the teacher and student roles?
(Who is doing the work)
Danielson
Kristin:  pare down, bullet list

14. Level 1: Recall & Reproduction Products
Fill in the blank tasks
Edit sentences
Identify/write sentence types
Highlight key words
Follow directions
Show and tell
Locate or recall quotes
Document/cite sources
Brainstorm
Identify parts of speech
Label or locate parts in a diagram
Vocabulary definitions
Decoding words
Level 1 tasks let us check for background knowledge or understanding

15. With your partner, create one or two examples of a DOK level 1 task.
Partner Task
With your partner, create one or two examples of a DOK level 1 task.
Kristin
(Picture)
Level 1 tasks are still important – they show students' understanding.  What's a level 1 task you use(or might use)in your class?

16. Level 1: Recall & Reproduction Examples
Describe an event, character, setting, etc. in a story
Write a list of key words you know about...
Recite/recall a fact or date related to ...
Write/retell in your own words ...
Cut out, draw, or match a picture that illustrates an event, process, or story
Retell in your own words/paraphrase
Use a dictionary, glossary, or thesaurus to find word meanings
Locate or retrieve information in verbatim form to answer a question
Edit applying a standard set of conventions and/or criteria that should eventually be automated (e.g., applying rules for grammar)

17. Level 2: Skill/Concept
Compare or differentiate among people, places, events, objects, text types
Apply multiple concepts when responding
Classify items into meaningful categories
Explain relationships
Provide and explain examples and non-examples
Amy 
Level 2 includes the engagement of mental processing beyond recalling, reproducing, or locating an answer. This level generally requires students to compare or differentiate among people, places, events, objects, text types, etc.; apply multiple concepts when responding; classify or sort items into meaningful categories; describe or explain relationships, such as cause and effect, character relationships; and provide and explain examples and non-examples. A Level 2 “describe or explain” task requires students to go beyond a basic description or definition to predict a possible result or explain “why” something might happen. The learner makes use of information provided in context to determine intended word meanings, which tools or approach is appropriate to find a solution (e.g., in a math word problem), or what characteristics to pay attention to when making observations. At this level, students are asked to transform/process target knowledge before responding. Example mental processes that often denote this particular level include: summarize, estimate, organize, classify, extend, and make basic inferences. 

18. Level 2: Skill/Concept Teacher Role Student Role
Questions to differentiate, infer, or check conceptual understanding
Models
Organizes/ reorganizes
Explores possible options or connections
Provides examples and non-examples 
Solves routine problems/tasks involving multiple decision points and concepts
Constructs models to show relationships
Demonstrates use of conceptual knowledge
Compiles and organizes
Illustrates/explains with examples or models
Examines

19. Level 2: Skill/Concept Possible Products
Captioned Photos Summary
Timeline
Demonstration
Presentation Interview
Journal Entry
Graphic Organizer
Relationship Mind Maps
Blog Commenting
Survey Development

20. With your partner, create one or two examples of a DOK level 2 task.
Partner Task
With your partner, create one or two examples of a DOK level 2 task.

21. Level 2: Skill/Concept Examples
Sequence a key chain of events and supporting details using a timeline, cartoon strip, outline or flow chart
Write a summary /informational report or develop an outline of central ideas and supporting details
Make a diorama to illustrate/explain an event
Create a questionnaire or survey to answer a question
Participate in a simulation in order to understand and describe differing perspectives
Write a diary/blog entry for a character or historical figure

22. Level 3: Strategic Thinking and Reasoning
Analysis
Evaluation
Explore Questions with Multiple Outcomes
Provide Evidence
Integration of conceptual knowledge and multiple skills
Focus on one item (i.e. text, investigation, key source)
Tasks and classroom discourse falling into this category demand the use of planning, reasoning, and higher order thinking processes, such as analysis and evaluation, to solve real-world problems or explore questions with multiple possible outcomes. Stating one’s reasoning and providing relevant supporting evidence are key markers of DOK 3 tasks. The expectation established for tasks at this level require an in-depth integration of conceptual knowledge and multiple skills to reach a solution or produce a final product. DOK 3 tasks and classroom discourse focus on in-depth understanding of one text, one data set, one investigation, or one key source, whereas DOK 4 tasks expand the breadth of the task using multiple texts or sources, or multiple concepts/disciplines to reach a solution or create a final product.

23. Level 3: Strategic Thinking and Reasoning
Teacher Role
Student Role
Questions to probe reasoning and underlying thinking
Asks open-ended questions
Provides criteria and examples
Encourages multiple approaches and interpretations 
Determines when/where (text, concept) depth and exploration is most appropriate 
Uncovers and selects  supporting evidence Critiques
Analyzes
Debates
Makes claims and judgments
Plans
Initiates questions
Argues
Sustains inquiry into topics or deeper problems
Applies to the real world 

24. Level 3: Strategic Reasoning Possible Products
Debate from a given perspective
Multi-paragraph essay or short story
Literary critique
Play, book, music, or movie review
Informational report with several subtopics
Fact-based argument (Is this criticism supported by the historical facts?)

25. Level 3: Strategic Reasoning Examples
Write a letter to the editor after evaluating a product
Use evidence to generate criteria for making judgments
Make a booklet or brochure about a topic, organization, or issue
Participate on a panel to discuss differing viewpoints on...
Prepare a speech to support your perspective about ...
Write an essay, short story, poem, or play
Propose an alternate solution to a problem studied

26. Which one of these tasks is a level 3 task? How do you know?
Product:  create a "trailer" video that includes evidence from the text:
Task A: Use a central theme that is expressed across all three texts to show how authors use symbolism and metaphor in developing their theme.
Task B: Summarize the story, including only the most important key events, characters, and settings involved in each stage of the story.
Task C: Explain the central theme of a text, showing the interrelationships between the characters and events in the story. 
Task C is the level 3 task!
Which level are the other 2?  Why?
Task A is level 4 –synthesize ideas across multiple texts,AND applies concepts taught
Task B is level 2 – only uses one story,summarizes
Level 3 & 2 are where we want to see most of our LA work in reading
In writing, level 4!

27. Level 4: Extended Thinking
Use of multiple texts/sources to reach a solution or create a final product
Use of multiple concepts/disciplines to reach a solution or create a final product
Sustain strategic thinking over a longer period of time
Authentic problems and audiences
Collaboration within a project-based setting
Curricular elements assigned to this level demand extended and integrated use of higher order thinking processes such as critical and creative-productive thinking, reflection, and adjustment of plans over time. Students are engaged in conducting multi-faceted investigations to solve real-world problems with unpredictable solutions. Employing and sustaining strategic thinking processes over a longer period of time to solve the problem or produce an authentic product is a key feature of curricular objectives assigned to DOK 4. Key aspects that denote this particular level typically include authentic problems and audiences, and collaboration within a project-based setting.
DOK 3 tasks and classroom discourse focus on in-depth understanding of one text, one data set, one investigation, or one key source, whereas DOK 4 tasks expand the breadth of the task using multiple texts or sources, or multiple concepts/disciplines to reach a solution or create a final product.

28. Level 4: Extended Thinking
Teacher Role
Student Role
Questions to extend thinking and broaden perspectives
Facilitates teaming, collaboration, self-evaluation
Designs
Takes risks
Researches synthesizing multiple resources
Collaborates
Plans, organizes, and modifies
Creates concrete tangible products

29. Level 4: Extended Thinking Possible Products
Short film/documentary
Agency presentation
Research report
Play
Essay
Newspaper or series of articles
Multi-Media product
Anthology of original writing, art, music, etc.

30. Level 4: Extended Thinking Activities
Applying information from more than one discipline to solve open- ended problems in new or real-world situations
Tasks that require multiple roles and collaboration and coordination with others (e.g., script writing, camera work, editing, and acting/ talent)
Tasks that involve drawing evidence from multiple sources to support solutions/conclusions 
Organizing/conducting a community service project or school-based event
Writing original pieces that apply skills and concepts taught

31. Team Task – Sort Again With your team, Resort into four categories.
Depth of Knowledge Level 1
Depth of Knowledge Level 2
Depth of Knowledge Level 3
Depth of Knowledge Level 4
Kristin
Did anything change from the first sort to now?

32. Partner Activity Choose a task from the sort Level it up!
Share with group
Pick Level 1 or 2 task from a card – make it a level 3 or 4
Model an example

33. Scaffolding Level 3 and 4 Tasks
Graphic Organizer
(show examples – how graphic organizers scaffold complex thinking
Theme pyramid)

34. Modify tasks from the curriculum and up the depth of knowledge
Time to work!
Modify tasks from the curriculum and up the depth of knowledge

35. Resources
Hess, K. (2013). A Guide for Using Webb’s Depth of Knowledge with Common Core State Standards. Retrieved from Evaluation-System/How-to-Design-and-Select-Quality- Assessments/Webbs-DOK-Flip-Chart.pdf.aspx

36. Contact us! Amy Allison, allisona@Issaquah.wednet.edu
Kristin Edlund,