1. Bigger Questions, Deeper Answers
David Irwin
Language Development Opportunities

2. Objectives
Apply all levels of the Depth of Knowledge scale to lessons, as appropriate Learn practical tools for assessing language use in content topics Develop student-self-assessment tools as appropriate for content, grade and language level by Applying formative assessment strategies to practice lessons and Planning lessons for your setting with DOK question strategies and language assessment tools embedded

3. Plan Plan a lesson with assessment tools embedded
Practice using embedded formative assessment tools in sample lessons
The assessment tools
Yes/no
True/false/evidence
Categorical/evidence
Reflective
Rubric (Likert scale)

4. Today’s Mission --- 3 products
A sheltered lesson plan for now or next year
Content and language objectives
Build background – connect it to something previous
Comprehensible input – visual, clear language
Interaction – academic conversation
Practice/application --- something hands-on
Assessment
A formative assessment for lesson(s)
A student self-assessment for that lesson
Then back at your school– share the lesson, use the tool(s), get the feedback from your Collaborative Expertise team

5. Questioning Activities

6. DOK: Stick it to your practice
Do you ask questions in each of the categories?
Do your students ask questions in each of the categories?
What are the strategies in language instruction to build better questioning skills in students?

7. Level 1: Recall and Reproduction
Bloom’s first two levels: Remembering and Understanding
Recall a fact, information, or procedure.

8. Level 1: Recall and Reproduction
Imperative format:
arrange, calculate, define, draw, identify, list, label, illustrate, match, measure, memorize, quote, recognize, repeat, recall, recite, state, tabulate, use, tell who‐ what‐ when‐ where- why
Question format:
Can you recall…?
When did…happen?
Who was…?
How can you recognize…?
What is …?
How can you find the meaning of …?
Can you recall …?
Can you select …?
How would you write …?
What might you include on a list about …?
Who discovered …?
What is the formula for …?
Can you identify …?
How would you describe …?

9. Level 2: Skill/Concept Bloom’s third level: Applying
Engages mental process beyond habitual response using information or conceptual knowledge. Requires two or more steps.

10. Level 2: Skill/Concept Imperative format:
apply, categorize, determine cause and effect, classify, collect and display, compare, distinguish, estimate, graph, identify patterns, infer, interpret, make observations, modify, organize, predict, relate, sketch, show, solve, summarize, use context clues
Question format:
Can you explain how…affected…?
How would you apply what you learned to develop …?
How would you compare …? Contrast?
How are…alike? Different?
How would you classify the type of …?
What would you use to classify …?
What can you say about …?
How would you summarize …?
What steps are needed to edit…?
When would you use an outline to …?
How would you estimate …?
How could you organize …?
What do you notice about …?

11. Level 3: Strategic Thinking
Bloom’s fourth level: Analyzing
Requires reasoning, developing plan or a sequence of steps, some complexity, more than one possible answer, higher level of thinking than the previous two levels.

12. Level 3: Strategic Thinking
Imperative format:
apprise, assess, cite evidence, critique, develop a logical argument, differentiate, draw, conclusions, explain phenomena in terms of concepts, formulate, hypothesize, investigate, revise, use concepts to solve non‐routine problems
Question format:
How is…related to…?
What conclusions can you draw?
What conclusion can be drawn from these three texts?
How would you adapt…to create a different…?
How would you test …?
Can you predict the outcome if…?
What is the best answer? Why?
What is your interpretation of this text? Support your rationale.
How would you describe the sequence of …?
What facts would you select to support …?
Can you elaborate on the reason …?
What would happen if …?
Can you formulate a theory for …?

13. Level 4: Extended Thinking
Bloom’s two highest levels: Evaluating and Creating
Requires investigation, complex reasoning, planning, developing, and thinking ‐ probably over an extended period of time.
*Longer time period is not an applicable factor if work is simply repetitive and/or does not require higher‐order thinking.

14. Level 4: Extended Thinking
Imperative format:
analyze, apply concepts, compose, connect, create, critique, defend, design, evaluate, judge, propose, prove, support, synthesize
Project format:
Apply information from one text to another text to develop a persuasive argument.
Write a thesis, drawing conclusions from multiple sources.
Design and conduct an experiment. Gather information to develop alternative explanations for the results of an experiment.
Write a research paper on a topic.
DOK 4 requires time for extended thinking.

15. Practicing DOK Questions
Choose a topic, write several questions at each level:
Growing flowers for sale
Maintaining a park, golf course or ball field
Selling a _____________________
Building a bridge over the Columbia River
Or…Pick a topic from your class, write questions at each level:
__________________________

16. Transferring the power: Students ask!
In your groups, brainstorm strategies to teach students to ask higher order questions
Assess:
What are your current question asking practices?
Who do you ask
How much language do you expect in the answer
How long do you wait
What are your students current question asking skills?
What is a reasonable next step for them? (I + 1)

17. Assessment OF Learning
Summative
High stakes
“10,000 ft” information on class, school or district
Broad planning
Not flexible for immediate planning
Teacher, district or publisher made

18. Assessment FOR Learning
Formative
Teacher, team, school or publisher made tools
Interspersed throughout lesson
Exit tasks
Informal or formal
Information on the class or individuals
Flexible

19. Assessment AS Learning
We build the tools ourselves
We use the tools
We discuss/evaluate the results
We keep or discard the records
We change/adapt the tools as needed
We don’t wait till the end, we don’t even wait for an official evaluator
Embedded as a part of the lesson
Gottlieb, M (2016) Assessing English Learners: Bridges to Educational Equity

20. Models of Tools Yes/No I can describe where the story takes place Yes
 I can tell when the story took place
I can describe who the characters are in the story
 I can tell what the characters did
 I can tell (describe) what happens (events) in the story
  I can tell (describe) what happens in the end (conclusion) of the story

21. Models of Tools Yes/No I can find a percentage of a whole Yes No
 I can find a percentage of a part of a whole
I can explain my thinking on how I find a percentage of a part of a whole
 I can calculate comparative percentages
 I can explain my thinking on how I calculate comparative percentages

22. True/False Evidence My answer before reading T/F What the text said
 My answer before reading T/F
 What the text said
What I learned
 Mt St Helens is a dormant volcano
 No one knew it was going to erupt in 1980
 No one know how much damage it might cause
 Everyone got away safely

23. Categorical/Evidence
Acceptable
Excellent
Evidence/Commentary
I completed three paragraphs
 My paper opens with a hook
 Each paragraph has a topic sentence
 Each topic is supported by at least three details
 My paper has a convincing closer

24. Reflective Designer (after dynamic/kinetic energy unit designing cars)
 Would you like to continue designing other vehicles? Anything else?
 How did you feel about your design?
 What was the research question (hypothesis) you were trying to answer?
 What did you find?
 What does your finding lead you to think about next?

25. Rubric I use lesson environmental terms from the word wall
 Not quite
Sometimes
Most of the time
Always
 I use lesson environmental terms from the word wall
 I compare/contrast positive and negative environmental changes
 I discuss causes and effects of environmental change in an organizer
 I talk to my family about ways to reduce, reuse and recycle
 I share my reduce, reuse and recycle data with my group

26. Rubric I can describe at least three key points of Social Darwinism
 Not quite
Sometimes
Most of the time
Always
 I can describe at least three key points of Social Darwinism
 I can compare Social Darwinism to original Darwinism
I can discuss the effects of Social Darwinism evident in our culture today
 I can make evaluative statements about Social Darwinism

27. References
Cummins, J. (1979) Cognitive/academic language proficiency, linguistic interdependence, the optimum age question and some other matters. Working Papers on Bilingualism, No. 19, Fisher, D. & Frey, N. (2007) Checking for Understanding: Formative Assessment Techniques for Your Classroom. ASCD: Alexandria, VA. Gottlieb, M. (2006) Assessing English Language Learners: Bridges from Language Proficiency to Academic Achievement. Corwin: Thousand Oaks, CA. Gottlieb, M. (2016) Assessing English Language Learners: Bridges to Educational Equity. Corwin: Thousand Oaks, CA. Joos, M. (1967) The Five Clocks: A Linguistic Excursion in the Five Styles of English Usage. Harcourt, Brace & World: New York. Krashen, S. (1982) Principles and practice in second language acquisition. Hayward, CA: Alemany Press. O’Malley, J.M. & Pierce, L.V. (1996) Authentic Assessment for English Language Learners: Practical Approaches for Teachers. Addison Wesley. Wiliam, D. & Leahy S. (2015) Embedding Formative Assessment: Practical Techniques for K-12 Classrooms. Learning Sciences International: West Palm Beach, FL.