1. Depth of Knowledge and Cognitive Demand QualityCore Professional Development Day 1, 2–1

2. Day 1, 2–2 Essential Question How can we know that our expectations of students are at a level of rigor that prepares them for the End-of-Course Assessment and for success in college or the workplace? Day 1, 2–2

3. Day 1, 2–3 By the conclusion of this segment, you will be able to:  Explain the differences between the cognitive levels of the depth-of-knowledge (DOK) framework  Analyze sample test items for depth of knowledge and compare those items across levels to differentiate the level of thinking required  Use the QualityCore Test Blueprint to review the percent of items found at each depth-of-knowledge level and discuss the classroom assessment implications of that distribution Learning Objectives

4. Day 1, 2–4  A framework for evaluating curriculum, standards, and assessments so they can be studied for alignment  Focuses on content and cognitive demand of test items, instructional strategies, and performance standards What is Depth of Knowledge?

5. Day 1, 2–5  Items need to measure the content described in the ACT Course Standards,  And they need to measure them at the performance levels specified,  So that the intended curriculum (e.g., as specified in ACT Course Standards) aligns with enacted curriculum and assessed curriculum (e.g., end-of-course tests). Alignment

6. Day 1, 2–6  An item measuring main idea may be at a low level of cognitive demand: ~What happened in this story?  Or it can be at a high level of cognitive demand: ~What is the theme of this passage? In English

7. Day 1, 2–7  First introduced in response to No Child Left Behind, where states were required to show that their standards, curriculum, and assessments were aligned with equivalent breadth and depth  Initially developed in collaboration with Council of Chief State School Officers (CCSSO)  Now widely used across the country The Depth-of-Knowledge Framework

8. Day 1, 2–8  Level 1 measures Recall at a literal level.  Level 2 measures a Skill or Concept at an interpretive level.  Level 3 measures Strategic Thinking at an evaluative level.  Level 4 measures Extended Reasoning. Webb (2002) QualityCore and DOK 4 Levels

9. Day 1, 2–9  Requires recall of information such as a fact, term, definition, or a simple procedure  Requires students to demonstrate a rote response or perform a simple procedure DOK Level 1 Recall: Literal

10. Day 1, 2–10  Find facts and details  Identify figurative language  Label simple parts of speech  Recall elements and details of passage structure (e.g., plot, sequence, setting, character)  Use appropriate grammar, punctuation, capitalization, and spelling DOK Level 1: English

11. Day 1, 2–11  Who is Hamlet’s mother?  What is the setting for the beginning of the play Hamlet?  What happens right after Laertes is fatally wounded? Sample Questions Level 1

12. Day 1, 2–12  Persuasive Language and Logic ~Locate important details and facts that support ideas, arguments, or inferences in increasingly challenging text...  Writing ~Correctly spell commonly misspelled/confused words Sample Level 1 ACT Course Standards

13. Day 1, 2–13  What is another ACT Course Standard that is Level 1?  What kinds of questions could you ask in class discussion or on an exam to reflect the Level 1 standard you found? Do You See Any Others?

14. Day 1, 2–14 15–25% of the questions on the English 9 End-of-Course Assessment are Level 1. http://qc.vantage.com/qualitycore/do/log English 9 Test Blueprint

15. Day 1, 2–15  Requires mental processing beyond recall or reproducing an answer  Students must make some decisions about how to approach a problem  Cognitive demands are more complex than in Level 1 DOK Level 2 Skill/Concept: Interpretive

16. Day 1, 2–16  Use context clues to determine the meaning of a word  Predict a logical outcome based on information in the story  Summarize major events, ideas, and pertinent details in a text  Use organizational strategies to structure written work DOK Level 2: English

17. Day 1, 2–17  What does the word vagabond mean in sentence 3 of the story?  What is likely to happen next, given the events described in the story?  What is the lesson of this fable?  What does the narrator reveal about her mother in the first paragraph?  Write a paragraph summarizing paragraphs 1–3 in the text. Sample Questions Level 2

18. Day 1, 2–18  Apply knowledge of Greek, Latin, and Anglo-Saxon affixes, inflections, and roots to understand unfamiliar words...  Summarize and paraphrase information... identifying key ideas...  Distinguish between fact and opinion Sample Level 2 ACT Course Standards

19. Day 1, 2–19  What is another ACT Course Standard that is Level 2?  What kinds of questions could you ask to reflect the standard? Do You See Any Others?

20. Day 1, 2–20 50–70% of the questions on the English 9 End-of-Course Assessment are Level 2. Test Blueprint for English 9

21. Day 1, 2–21  Requires planning, thinking, explaining, justifying, using evidence, conjecturing, and postulating  Cognitive demands are complex and abstract DOK Level 3 Strategic Thinking: Evaluative

22. Day 1, 2–22  Describe author’s purpose and how it affects the interpretation of text  Analyze and describe the characteristics of a passage  Infer across an entire passage  Identify abstract themes  Analyze and synthesize information  Use voice appropriate to the purpose and audience  Edit writing to produce a logical progression of ideas DOK Level 3: English

23. Day 1, 2–23  How does Edgar Allen Poe create a feeling of suspense in his poem “The Raven”? How does this differ from the way Simenon creates suspense in his story about Inspector Maigret?  What makes John a true tragic figure in this passage?  What does the author do to make you believe Mary’s story?  Rewrite this story from the point of view of the brother. Sample Questions Level 3

24. Day 1, 2–24  Explain the effects of the author’s life upon his or her work (e.g., Alexander Solzhenitsyn’s experience in the gulag as reflected in his novel One Day in the Life of Ivan Denisovich)  Critique the treatment and scope of ideas from multiple sources on the same topic... Sample Level 3 ACT Course Standards

25. Day 1, 2–25  What is another ACT Course Standard that is Level 3?  What kinds of questions could you ask to reflect the standard? Do You See Any Others?

26. Day 1, 2–26 15–25% of the questions on the English 9 End-of-Course Assessment are Level 3. Test Blueprint for English 9

27. Day 1, 2–27  Requires complex reasoning, planning, developing, and thinking  Usually requires extended time  Requires higher-order thinking DOK Level 4 Extended Thinking

28. Day 1, 2–28 Students take information from at least one passage and are asked to apply this information to a new task. They may also be asked to develop hypotheses and perform complex analyses of the connections among texts. Some examples that represent but do not constitute all of Level 4 performance are:  Analyze and synthesize information from multiple sources.  Examine and explain alternative perspectives across a variety of sources.  Describe and illustrate how common themes are found across texts from different cultures. Level 4 Reading

29. Day 1, 2–29 The standard at this level is a multiparagraph composition that demonstrates synthesis and analysis of complex ideas or themes. There is evidence of a deep awareness of purpose and audience. For example, informational papers include hypotheses and supporting evidence. Students are expected to create compositions that demonstrate a distinct voice and that stimulate the reader or listener to consider new perspectives on the addressed ideas and themes. An example that represents but does not constitute all of Level 4 performance is:  Write an analysis of two selections, identifying the common theme and generating a purpose that is appropriate for both. Level 4 Writing

30. Day 1, 2–30 What kinds of assignments and standards are Level 4? Level 4 Assignments

31. Day 1, 2–31  Find the sample test items in your Participant Handbook.  Fold over the page with the key for now.  Take the test and indicate the depth-of- knowledge level of each question.  Check your answers against the key. DOK and Assessment

32. Day 1, 2–32 What are three ways you can use the depth-of-knowledge framework to increase the rigor of your instruction, your assignments, and your assessments? How Can You Use This Framework?