1. When It Rains Rigor The puddles are deeper!
Welcome…..
Rigor – the mysterious word – can seem confusing and daunting.
What is it really all about?

2. and the Depth of Knowledge
Bringing Clarity to ….
Our goal today is to clarify rigor within the concept of Depth of Knowledge or DOK.
Cognitive Rigor
and the Depth of Knowledge

3. Cognitive Rigor Overview
Part 1
Cognitive Rigor Overview
Conceptualizing Cognitive Rigor
Dddd
Part 1
Your Definition of Cognitive Rigor and Your Questions
Different Models of Rigor
Development of the Cognitive Rigor (CR) Matrix
Revisit Your Definition of CR and Questions

4. Rigor is…. Clarifying Think About Rigor Applying Rigor
Dr. Hess
Clarifying
Think About Rigor
Rigor is….
Handout 1
Applying Rigor
A basic comprehension question...
A rigorous comprehension question…
Handout #1 Make… “Your class has just read a version of Goldilocks and the Three Bears. What is a basic comprehension question you might ask? What is a more rigorous comprehension questions you might ask?” Have attendees complete Handout #1 and share with a peer. “We will come back to your rigor definition and questions later on in this presentation.”

5. Different Models of Rigor
There are different models describing cognitive rigor. Each model actually addresses something different.
Bloom – What type of thinking (verbs) is needed to complete a task?
Webb – How deeply do you have to understand the content to successfully interact with it? How complex is the content?
Hess combined Dr. Norman Webb’s Depths of Knowledge and Dr. Benjamin Bloom’s Bloom’s Cognitive Taxonomy to develop the Cognitive Rigor (CR) Matrix.
Different schools, districts, teachers and states may use different models for higher leveled instruction.

6. The “old” and “new” Blooms
Bloom’s Taxonomy [1956 ] & Bloom’s Cognitive Process Dimensions [2005]
Knowledge—Define, duplicate, label, list, name, order, recognize, relate, recall
Remember—Retrieve knowledge from long-term memory, recognize, recall, locate, identify
Comprehension—Classify, describe, discuss, explain, express, identify, indicate, locate recognize, report, review, select, translate
Understand—Construct meaning, clarify, paraphrase, represent, translate, illustrate, give examples, classify, categorize, summarize, generalize, predict…
Application—Apply, choose, demonstrate, dramatize, employ, illustrate, interpret, practice, write
Apply—Carry out or use a procedure in a given situation; carry out or use/apply to an unfamiliar task
Analysis—Analyze, appraise, explain, calculate, categorize, compare, criticize discriminate, examine
Analyze—Break into constituent parts, determine how parts relate
Synthesis—Rearrange, assemble, collect, compose, create, design, develop, formulate, manage, write
Evaluate—Make judgments based on criteria, check, detect inconsistencies/fallacies, critique
Evaluation—Appraise, argue, assess, choose, compare, defend, estimate, explain, judge, predict, rate, core, select, support, value
Create—Put elements together to form a coherent whole, reorganize elements into new patterns/structures
“We are most familiar with the Blooms Taxonomy. The “old” Blooms developed in 1956 presents first with nouns
Such as Knowledge and then leads into verbs that represent knowledge. The new Blooms begins with a verb
“remember” replacing the noun “knowledge,” and then moves into verbs written within a context of a partial task.
Evaluation is replaced as the highest level of difficulty with Synthesis – or now the verb “create.”
This is important because researchers began to recognize that just nouns and/or verbs alone are not distinct enough of a standard to distinguish what is truly higher level thinking.

7. Hess Cognitive Rigor Matrix
DOK Level
Level Descriptors 1
Recall and Reproduction 2
Skills and Concepts 3
Strategic Thinking and Reasoning 4
Extended Thinking
Hess Cognitive Rigor Matrix
Webb’s 
Depth of Knowledge
DOK LEVEL 1
Recall and Reproduction
DOK LEVEL 2
Basic Skills and Concepts
DOK LEVEL 3
Strategic Thinking and Reasoning
DOK LEVEL 4
Extended Thinking
Blooms 
Remember
(Knowledge)
Retrieve knowledge from long-term memory, recognize, recall, locate, identify.
Ka Recall, recognize, or locate basic facts, details, events, or ideas explicit in texts.
Kb Read words orally in connected text with fluency & accuracy.
Kc-Define terms.
Understand
(Comprehend)
Construct meaning, clarify, paraphrase, represent, translate, illustrate, give examples, classify, categorize, summarize, generalize, infer a logical conclusion), predict, compare/contrast, match like ideas, explain, construct models.
Cd Identify or describe literary elements (characters, setting, sequence, etc.)
Ce Select appropriate words when intended meaning/definition is clearly evident.
Cf Describe/explain who, what, where, when, or how.
Ch Specify, explain, show relationships; explain why, cause-effect.
Give non-examples -examples.*
Ci Summarize results, concepts, ideas.
Cj Make basic inferences or logical predictions from data or texts.
Ck Identify main ideas or accurate generalizations of texts.
Cl Locate information to support explicit-implicit central ideas.
Cu Explain, generalize, or connect ideas using supporting evidence (quote, example, text reference).
Cv Identify/ make inferences about explicit or implicit themes.
Cw Describe how word choice, point of view, or bias may affect the readers’ interpretation of a text.
CK Explain how concepts or ideas specifically relate to other content domains or concepts.
CL Develop generalizations of the results obtained or strategies used and apply them to new problem situations.
Apply
Carry out or use a procedure in a given situation; carry out (apply to a familiar task), or use (apply) to an unfamiliar task.
APgUse language structure (pre/suffix) or word relationships(synonyms/antonym) to determine meaning.
APm Use context to identify word meanings
APn Obtain and interpret information using text features.
APx Use concepts to solve non-routine problems.
Devise an approach among many alternatives to research a novel problem.*
APM Illustrate how multiple themes (historical, geographic, social) may be interrelated.
Analyze
Break into constituent parts, determine how parts relate, differentiate between relevant-irrelevant, distinguish, focus, select, organize, outline, find coherence, deconstruct (e.g., for bias or point of view).
ANo Identify whether specific information is contained in graphic representations (e.g., map, chart, table, graph, T-chart, diagram) or text features (e.g., headings, subheadings, captions).
ANp Categorize/compare literary elements, terms, facts, details, events.
ANq Identify use of literary devices.
ANr Analyze format, organization, & internal text structure (signal words, transitions, semantic cues) of different texts.
ANs Distinguish: relevant-irrelevant information; fact/opinion.
ANt Identify characteristic text features; distinguish between texts, genres.
ANy Analyze information within data sets or texts.
ANz Analyze interrelationships among concepts, issues, and problems.
ANA Analyze or interpret author’s craft (literary devices, viewpoint, or potential bias) to critique a text.
ANB Use reasoning, planning, and evidence to support inferences.
ANN Analyze multiple sources of evidence, or multiple works by the same author, or across genres, time periods, themes.
ANO Analyze complex/abstract themes, perspectives, concepts.
ANP Gather, analyze, and organize multiple information sources.
ANQ Analyze discourse styles.
Evaluate
Make judgments based on criteria, check, detect inconsistencies or fallacies, judge, critique.
EVC Cite evidence and develop a logical argument for conjectures.
EVD Describe, compare, and contrast solution methods.
EVE Verify reasonableness of results.
EVF Critique conclusions drawn.
EVR Evaluate relevancy, accuracy, & completeness of information from multiple sources.
EVS Draw & justify conclusions
EVT Apply understanding in a novel way; provide argument or justification for the application.
Create
(Synthesize)
Reorganize elements into new patterns/structures, generate, hypothesize, design, plan, produce.
SYG Generate conjectures or hypotheses based on observations or prior knowledge and experience.
SYH Synthesize information within one source or text.
SYI Develop a complex model for a given situation.
SYJ Develop an alternative solution.
SYU Synthesize information across multiple sources or texts.
SYV Articulate a new voice, theme, knowledge or perspective.
Norman Webb is credited for making the distinction between the level of difficulty (Blooms) and the depth or complexity of the content students needed to successfully interact with. His process of analyzing the cognitive expectation demanded of standards is known as Depth of Knowledge or DOK.
Hess Quote: “The Hess Cognitive Rigor Matrix (CRM) is a tool that was developed to enhance assessment planning and instructional practices at the classroom level (Hess, Carlock, Jones, & Walkup, 2009). The CRM superimposes two different cognitive complexity frameworks – Bloom’s (Revised) Taxonomy and Webb’s Depth of Knowledge levels – to produce a means of analyzing the emphasis placed on curricular materials, instructional focus, and classroom assessment.”

8. How Does Rigor Look in the Classroom?
Article: What exactly do “fewer, clearer, and higher standards” really look like in the classroom? Using a cognitive rigor matrix to analyze curriculum, plan lessons, and implement assessments (Hess, Carlock, Jones, & Walkup, 2009)
What “take-away” from each section of the article can we apply to classroom instruction?
Section 1: “Take-Aways,” from Beginning with Bloom
Section 2 : “Take-Aways,” from Webb’s Depth-of-Knowledge Levels
Section 3: “Take-Aways,” from Cognitive Rigor and the CR Matrix
Handout #2 Article: Before beginning activity– have everyone read the entire article.
Group into school teams. Have each team focus on one of the 3 sections of the article.
Then, come back together and have each team share their “take-aways,” and fill out the
Comment section of Handout # 1: How has your definition of rigor changed? How?

9. Determining Depth-of-Knowledge Levels
The intended student learning outcome determines the DOK level. What mental processing must occur?  
While verbs may appear to point to a DOK level, it is what comes after the verb that is the best indicator of the rigor/DOK level.
Describe the physical features of a plant.
Describe how the two political parties are alike and different.
Describe the most significant effect of WWII on the nations of Europe.
If there is only one correct answer, it is probably level DOK 1 or DOK 2  
DOK 1: you either know it (can recall it, locate it, do it) or you don’t
DOK 2 (conceptual): apply one concept, then make a decision before going on applying a second concept
If more than one solution/approach, requiring evidence, it is DOK 3 or 4
DOK 3: Must provide supporting evidence and reasoning (not just HOW solved, but WHY – explain reasoning)
DOK 4: all of “3” + use of multiple sources or texts.
Dr. Hess
The intended student learning outcome determines the DOK level. What mental processing must occur?
Do a quick share out “what do you think this means….?” Lead attendees to think about when they can’t be sure what DOK level a question or task it – ultimately it comes down to the intended learning outcome.
These are “rules of thumb,” from Dr. Hess (in speech bubbles)… We will go more into depth (pardon
The pun ) about DOK, but for now lets go back and look at the 2 questions you wrote for comprehension.
Handout #3 You wrote a basic and a more rigorous question from Goldilocks and the 3 Bears. This
Lesson plan template could be used in many ways, but this is one example. Mark where your two questions would best fit on the lesson plan template.
Was their an “eye-opener,” for you? Talk to a partner – then share as a team!
Rules of Thumb

10. Note: not a complete matrix – an abbreviated example
The Hess Cognitive Rigor Matrix: Applies Webb’s DOK to Bloom’s Cognitive Process Dimensions
Note: not a complete matrix – an abbreviated example
Depth +
thinking
Level 1 Recall & Reproduction
Level 2 Skills & Concepts
Level 3
Strategic Thinking/ Reasoning
Level 4
Extended Thinking
Remember
-Recall, locate basic facts, details, events
Not appropriate at this level
Understand
-Select appropriate words to use when intended meaning is clearly evident
-Specify or explain relationships
-summarize
-identify central idea
-Explain, generalize, or connect ideas using supporting evidence (quote, example…)
-Explain how concepts or ideas specifically relate to other content domains or concepts
Apply
-Use language structure (pre/suffix) or word relationships (synonym/antonym) to determine meaning
-Use context to identify meaning of word
-Obtain and interpret information using text features
-Use concepts to solve non-routine problems
-Devise an approach among many alternatives to research a novel problem
Analyze
-Identify whether information is contained in a graph, table, etc.
-Compare literary elements, terms, facts, events
-analyze format, organization, & text structures
-Analyze or interpret author’s craft (literary devices, viewpoint, or potential bias) to critique a text
-Analyze multiple sources
-Analyze complex/abstract themes
Evaluate
-Cite evidence and develop a logical argument for conjectures
-Evaluate relevancy, accuracy, & completeness of information
Create
-Brainstorm ideas about a topic
-Generate conjectures based on observations or prior knowledge
-Synthesize information within one source or text
-Synthesize information across multiple sources or texts
Handout #3 Front
This is an abbreviated handout of Hess’ Cognitive Rigor Reading Matrix. We used the completed matrix
to develop the reading learning progressions and to write our assessment questions on the pre-assessments
and CFAs in order to match each standard to its correct DOK level demand.

11. The CR Matrix Lesson Plan Template Example Back to Goldilocks and the Three Bears
Depth +
thinking
Level 1 Recall & Reproduction
Level 2 Skills & Concepts
Level 3
Strategic Thinking- Reasoning
Level 4
Extended Thinking
Remember
-Recall facts
Understand
-Identify characters, setting, etc.
-Retell or summarize…
Apply
Analyze
-Compare-contrast
-Analyze multiple texts/sources & using text evidence for support
Evaluate
-Justify judgments using details/evidence from text
Create
-Develop a creative summary
Handout #3 Back
Activity: This is an example of how a lesson map could look for question and task development of Goldilocks and the 3 Bears. Look at the two questions you wrote at the beginning of this session. Write your questions on the matrix where you think they would align. Where would your basic comprehension question go?
Where would your more rigorous question go? Would you make any changes?

12. This is the end of Part 1: The Cognitive Rigor Overview
As we begin to think more about the implications of rigor and the Depths of Knowledge in our daily instruction, we can use the Cognitive Rigor (CR) Matrix to align lesson plans and write questions at higher DOK Levels.
In Part II of this training we will be:
Focusing on the DOK Descriptors on the CR Matrix
Learning how to Recognize and Write Questions at specific DOK Levels
Developing a tool for student use in the classroom