1. Mapping Rigor Putting the Pieces Together Presented by April 23, 2013
Elin Reuben, New Hanover County Schools
Karma Maples, New Hanover County Schools
Shelley Hollingsworth, Wake County Schools

2. Agenda 9:30 – 9:35 Welcome 9:35- 9:40 Introductions
9:40 – 10:00 Background
10:00 – 10: Introduction of DOK, RBT and rigor matrix
10:20- 10: Defining Rigor
10:25- 10: Exercise – Little Red Riding Hood
10:35- 10:50 What does Rigor look like?  
10:50 – 11: Rigor Review
11:20- 11:30 Group discussion
11:30- 11: Rigorous Assessments
11:50- 12:15 Creating assessment
12:15 – 12: Group discussion and sharing
12:25- 12: Exit survey

3. Intended Outcomes: Define rigor Understand DOK and RBT
Study and Use the Karen Hess’s Rigor Matrix
Applying rigor tool to evaluate units/lessons
Understand rigorous assessments
Collaborate and Reflect

4. Review of Previous Year
Evaluated where current units fit with the Common Core – participants brought units
Looked at misconceptions in relation to the Common Core and gifted education
Plugged units into Revised Bloom’s Matrix

5. Box of Chocolates
You have been hired as a designer to market packages of chocolate candy for Valentine’s Day. The manufacturer would like rectangu1ar boxes that do not have a top. The candy will be covered with colorful plastic wrap so the consumer can see the actual product. Use your math skills to master this task.
Directions:
1. Use 1 inch graph paper (that was printed on card stock) to create a template for your box.
2. Flip the paper over and decorate the package. (The printed squares should be inside the box.)
3. Assemble the box with clear tape.
Now it is time to do the math.
Answer the foil owing questions.
1. What are the dimensions of the box? (length, width, and height)
2. What is the total surface area of the box?
3. If the material used to construct the box costs 4 cents per square inch, how much did it cost the manufacturer to make the box?
4. How many 1 inch cubes of chocolate fudge will the box hold? (volume)
5. If it cost 20 cents to make each piece of candy, how much would it cost to fill the box with chocolates?
6. What is the manufacture’s total cost of goods (box materials and candy)?
7. If the consumer’s price is the manufacture’s total cost plus 20%, what will the actual price of the candy?
8. How much profit will the manufacturer make on each box that is sold?
5.MD.5.C
5.NBT.7
5.MD.3 & 4 & 5
5.NBT.7

6. Blank Revised Bloom’s

7. Example Bloom’s Matrix

8. What is rigor?
Let’s talk

9. How the Experts Define Cognitive Rigor…
Rigor is …
Quality of thinking, not quantity, and can occur in any grade at any subject (Bogess, 2007)
Deep immersion in a subject and should include real-world settings (Washor & Majkowki, 2006)
Thoughtful analysis with sufficient attention to accuracy and detail (Beane, 2001)
Helping students develop the capacity to understand content that is complex, ambiguous, provocative, and personally or emotionally challenging (Strong, Silver, & Perrini, 2001)

10. Developing a Cognitive Rigor Matrix
Currently, we use two different tools to describe cognitive rigor. Each addresses something different.
RBT (Revised Bloom’s Taxonomy) – What type of thinking (verbs) is needed to complete a task?
DOK (Webb’s Depth of Knowledge) – How deeply do you have to understand the content to successfully interact with it? How complex or abstract is the content?

11. Bloom’s Taxonomy (1956) & Bloom’s Cognitive Process Dimensions (2005)
Remember– Retrieve knowledge from long-term memory, recognize, recall, locate, identify
Understand – Construct meaning, clarify, paraphrase, represent, translate, illustrate, give examples, classify, categorize, summarize, generalize, predict
Apply – Carry out or use a procedure in a given situation; carry out or use/apply to an unfamiliar task
Analyze – Break into constituent parts, determine how parts relate
Evaluate – Make judgments based on criteria, check, detect inconsistencies/fallacies, critique
Create – Put elements together to form a coherent whole, reorganize elements into new patterns/structures.
Knowledge– Define, duplicate, label, list, name, order, recognize, relate, recall
Comprehension—Classify, describe, discuss, explain, express, identify, indicate, locate, recognize, report, review, select, translate
Application– Analyze, choose, demonstrate, dramatize, employ, illustrate, interpret, practice, write
Analysis– Analyze, appraise, explain, calculate, categorize, compare, criticize, discriminate, examine
Synthesis– Rearrange, assemble, collect, compose, create, design, develop, formulate, manage, write
Evaluation– Appraise, argue, assess, choose, compare, defend, estimate, explain, judge, predict, rate, core, select, support, value

12. Webb’s Depth-of-Knowledge Levels
What is DOK?
Let’s watch a video
Webb’s Depth-of-Knowledge Levels
DOK-1 – Recall & Reproduction – Recall of a fact, term, principle, concept; perform a routine procedure; locate details
DOK-2 – Basic Application of Skills/Concepts – Use of information; conceptual knowledge; select appropriate procedures for a given task; two or more steps with decision points along the way; routine problems; organize/ display data; interpret/use simple graphs; summarize; identify main idea; explain relationships; make predictions
DOK-3 – Strategic Thinking – Requires reasoning, or developing a plan or sequence of steps to approach problem; requires decision making or justification; abstract, complex or non-routine; often more than one possible answer; support solutions or judgments with text evidence
DOK-4 – Extended Thinking – An investigation or application to real world; requires time to research, problem solve, and process multiple conditions of the problem or task; non-routine manipulations; synthesize information across disciplines/content areas/multiple sources.

14. DOK is about depth & complexity –
Not difficulty!
The intended student learning outcome determines the DOK level. What mental process must occur?
While verbs may appear to point to a DOK level, it is what comes after the verb that is the best indicator of the rigor/DOK level.
Describe the process of photosynthesis
Describe how the two political parties are alike and different
Describe the most significant effect of WWII on the nations of Europe

16. Hess’s Cognitive Rigor Matrix:
Applies Webb’s DOK to RBT Cognitive Process Dimensions
Depth + Thinking
Level 1
Recall & Reproduction
Level 2
Basic Skills &
Concepts
Level 3
Strategic Thinking & Reasoning
Level 4
Extended
Thinking
Remember
Recall, locate basic facts, details, events
Understand
Select appropriate words to use when intended meaning is clearly evident
Specify, explain relationships, summarize, identify main ideas
Explain, generalize, or connect ideas using supporting evidence
Explain how concepts or ideas specifically relate to other content domains or concepts
Apply
Use language structure or word relationships to determine meaning
Use contest to identify meaning of word, obtain and interpret information using text features
Use concepts to solve non-routine problems
Devise an approach among many alternatives to research a novel problem
Analyze
Identify whether information is contained in a graph, table, etc.
Compare literacy elements, terms, facts, events, analyze format, organization, & text structures
Analyze or interpret author’s craft to critique a text
Analyze multiple sources or texts
Analyze complex/ abstract themes
Evaluate
Cite evidence and develop a logical argument for conjectures
Evaluate relevancy, accuracy, & completeness of information
Create
Brainstorm ideas about a topic
Generate conjectures based on observations or prior knowledge
Synthesis information within one source of text
Synthesize information across multiple sources or texts

17. Where do these questions fit into the matrix?
Story: Little Red Riding Hood
Where do these questions fit into the matrix?
1. Retell or summarize the story in your own words.
2. What was the story’s setting?
3. What is your opinion about the intelligence of the wolf? Justify using
details/evidence from the story.
4. What color was Red’s cape?
5. Identify transitional words and phrases that helped you know the
sequence of events in the story.
6. Write a telephone conversation between Little Red Riding Hood and her
mother that explains the wolf incident.
7. What did the wolf do to trick Little Red Riding Hood?
8. What is the same and different about your grandmother’s house and the house in the story?
9. Who was the main character?
10. Are all wolves (in literature) like the wolf in this story? Support your response using evidence from other texts.
11. How did the story end?
12. Where was Little Red Riding Hood going?
13. What are some of the examples of personification used in the story?

18. Hess CR Matrix
Write the number for each set of questions in the correct cell in the CR Matrix
Depth + Thinking
Level 1
Recall & Reproduction
Level 2
Basic Skills &
Concepts
Level 3
Strategic Thinking & Reasoning
Level 4
Extended
Thinking
Remember
Understand
Apply
Analyze
Evaluate
Create

19. Remember Understand Apply Analyze Evaluate Create Depth + Thinking
Level 1
Recall & Reproduction
Level 2
Basic Skills &
Concepts
Level 3
Strategic Thinking & Reasoning
Level 4
Extended
Thinking
Remember
(4) What color was Red’s cape?(12) Where was little red riding hood going?(7) What did the wolf do to trick Little Red Riding Hood? (11)How did the story end?
Understand
(9) Who was the main character? (2) What was the story’s setting?
(1) Retell or summarize the story in your own words.
Apply
(5) Identify transitional words and phrases to help you know the sequence of events in the story.
Analyze
(13) What are some examples of personification in the story? (8)What is the same and different about your grandmother’s house and the house in the story?
(10) Are all wolfs (in literature) like the wolf in this story? Support your response using evidence from other texts.
Evaluate
(3) What is your opinion about the intelligence of the wolf? Justify using details/evidence from the story.
Create
(6) Write a telephone conversation between Little Red Riding Hood and her mother that explains the wolf incident.

20. What happens when you
RIGOR

22. Kindergarten Antiques Road Show
Math Games

23. Is it rigor?

24. Resources
Google Site

25. Rigor in Curriculum
WCPSS has a tool called CMAPP which serves as an instructional guide
Tiered activities to include all levels of students
Efforts to align CMAPP with Common Core Standards started this year

26. Using DOK on a Daily Basis
DOK charts in student notebooks
Students take ownership and understand their own learning process
Natural differentiation
Assignments directly using DOK
Creating study guides
Make an assessment & answer it

27. Revised Bloom’s in the Classroom
WCPSS provides posters for all classrooms
Reminder to students and teachers
Consistency of language
Encouraged to reference frequently to students
Increased self-efficacy in students
Helps to see growth in individuals & curriculum

28. Is DOK different from Revised Bloom’s Taxonomy?
Depth of Knowledge (DOK) is a scale of cognitive demand. The Depth of Knowledge is NOT determined by the verb, but the context in which the verb is used and the depth of thinking required. See example below:
DOK 3-Describe a model that you might use to represent the relationships that exist within the rock cycle. (Requires deep understanding of rock cycle and a determination of how best to represent it)
DOK 2-Describe the difference between metamorphic and igneous rocks. (Requires cognitive processing to determine the differences in the two rock types)
DOK 1-Describe three characteristics of metamorphic rocks. (Simple recall)
Same verb—used at all three DOK levels
Focus of DOK
DOK requires looking at the assessment item/standard in order to determine the level.
DOK is about the item/standard and not the students’ ability.
The context of the assessment item/standard must be considered to determine the DOK-not just a look at what verb was chosen

29. Rigorous Assessments Days of definition tests and T/F are OVER!
Use DOK/RB to create questions: classroom and standardized
Better matches differentiation & rigor in curriculum
MSL/EOG Training – Major Shift in Thinking!
Are all my “Good Questions” Garbage??!!
Question Makeovers

30. Short List of Do’s & Don’ts for Stems
Stems should:
Be reasonably short and contain no extra info
Ask about only one objective
Not contain the word “not”
Avoid “of the following”
Avoid biased topics, slang words, etc.

31. Short List of Do’s & Don’ts for Foils
Foils should:
Avoid “none/all of the above”
“a & c only”
etc.
Have four plausible answers
Have answers nearly the same length
Avoid “crazy” answers or distracters

32. Sample EOG Question
How would smoking tobacco most likely have a negative impact on the ability of an individual to succeed at physical activities?
It interferes with balance & coordination
It decreases the ability to make decisions rapidly
It decreases stamina and cardiovascular efficiency
It decreases the speed & power of muscle contractions.

33. Old Question What is a physical change?
A change that produces a new substance
A change that does not produce a new substance
A change where atoms are rearranged
None of the above

34. Better? More rigorous? Or not?
A teacher does a demonstration for her class with a piece of notebook paper. Using a balance, she discovers that the mass of the paper is 20 g. She then takes the piece of paper and cuts it into several small pieces. The cumulative mass of the small pieces is 20 g. What have the students observed?
a chemical change involving change of shape and size of
matter
B. a chemical change involving change of mass
C. a physical change involving change of shape and size of
D. a physical change involving change of mass

35. Is this a rigorous question?
A paleontologist is giving a presentation about her research that theropod dinosaurs like Tyrannosaurus Rex walked more upright than bent over. She has a photo from her recent field research of a theropod track. How do you think she plans to use the photograph and diagram with Theropod A and Theropod B to support her theory?

36. Questions

37. Shelley Hollingsworth
Contact Information
Shelley Hollingsworth
Heritage Middle School
Karma Maples
Codington Elementary School
ext. 250
Elin Reuben
Roland-Grise Middle School
ext. 718