1. Instructional Critical Thinking Assessment Item Development
Thinking about Thinking: Methods to Assess Critical Thinking Instruction in Higher Education
Demonstration of
Instructional Critical Thinking Assessment Item Development
Linda Lynch, PhD, Educational Researcher, United States Army

2. Objectives Define Critical Thinking
Describe how to recognize CT performance
Discuss goals of an evaluator who must assess instructor CT integration
Describe performances associated with CT
Practice developing a performance measure of CT

3. Why is Critical Thinking Important?
Jean Piaget (1923) said that “the second most important goal of education is to form minds which can be critical, can verify, and not accept everything what is offered”, and….
Piaget, J (1932). The moral judgment of the child. London, UK: Kegan Paul.

4. Why is Critical Thinking Important?
…the first goal was to “create men and women who are capable of doing new things, not simply repeating what other generations have done – men who are creative, inventive, and discoverers”.

5. What Does Critical Thinking Look Like?

6. ?

7. What are Your Goals as a Program Evaluator Supporting Critical Thinking?
What did you want to learn?
What are your goals?
What would you like to take away from this demonstration?

8. Suggested Goals as a Program Evaluator Supporting Critical Thinking

9. Suggested Goals as a Program Evaluator Supporting Critical Thinking
1. Provide evidence that instructors are integrating Critical Thinking into instruction

10. Your Goals as a Program Evaluator Supporting Critical Thinking
1. Provide evidence that instructors are integrating Critical Thinking into instruction 2. YOUR ULTIMATE GOAL: Define performances that demonstrate quality CT instruction

11. How to Start Developing Performance Measures of CRITICAL THINKING INSTRUCTIONAL QUALITY?

12. Evaluators ask these questions…

13. Does the instructor demonstrate …

14. Universal Intellectual Standards

15. Universal Intellectual Standards
Accuracy:
Clarity:
Relevance:
Logic:
Sufficiency:
Precision:

16. Universal Intellectual Standards
Accuracy: Is that really true?
Clarity: Could you elaborate?
Relevance: How is that connected?
Logic: Does that make sense?
Sufficiency: Does that provide reason for all the outcomes?
Precision: Could you give more details?

17. Universal Intellectual Standards
Significance:
Fairness:
Breadth:
Depth:

18. Universal Intellectual Standards
Significance: Is this the most important problem to consider?
Fairness: Do I have a vested interest in the outcome?
Breadth: Should another opinion be considered?
Depth: Am I taking into account the most significant factors?

19. Does the instructor demonstrate …

20. Essential Intellectual Traits

21. Essential Intellectual Traits
Intellectual Humility: Sensitivity to bias prejudice and one’s own viewpoint
Intellectual Autonomy: A commitment to analyzing and evaluating beliefs based on reason
Fairmindedness: The need to treat all viewpoints alike
Intellectual Courage: The need to face ideas need beliefs toward which there is strong negative emotion

22. Essential Intellectual Traits
Confidence in Reason: The interests of humankind is served by encouraging the development of thinking skills
Intellectual Perseverance: The need to use intellectual truths in spite of opposition
Intellectual Empathy: The ability to understand the reasoning of others
Intellectual Integrity: The need to be true to one’s own thinking

23. Does the instructor demonstrate …

24. Elements of Reasoning
Purpose: All reasoning has a purpose that is stated clearly Questions: All reasoning is an attempt to answer a question or solve a problem Assumptions: All reasoning is based on Assumptions Points of View: All reasoning is accomplished from a point of view Information: All reasoning is based on Data and Information

25. Elements of Reasoning
Inferences: All reasoning contains inferences and interpretations Concepts: All reasoning is expressed through and shaped by concepts and ideas Implications: All reasoning has implications and consequences

26. Critical Thinking Model
Refer to Paul and Elder
Critical Thinking Model
Model: University of Louisville: Ideas to Action: Using Critical Thinking to Foster Student Learning and Community Engagement

27. Review the Elements of Thought

28. Dr. Richard Paul & Dr. Linda Elder The Foundation and Center for Critical Thinking Paul, R Elder, Linda (2008) The Miniature Guide to Critical Thinking Concepts and Tools. Foundation for Critical Thinking Press

29. Through speech, support, prompting, or discussion.
What is the instructor’s demonstration of critical thinking?
Through speech, support, prompting, or discussion.

30. Group Directions

31. GROUP Directions
Develop an assessment item that demonstrates the performance of an Intellectual Standard or Trait by an instructor 5 minutes

32. What Did You Choose to Measure? What did you Assess?
Group answers

34. Instructor Performance Instructor Behavior
What Should Be Measured or Assessed to Demonstrate Quality CT Instruction?
Instructor Performance
Instructor Behavior

35. What is an Example of an Instructor Performance Measure of Critical Thinking?

36. CT Performance Measures
Assessment item responses suggest a level of performance

37. Example Performance Measure
Does not allow students to make mistakes – hinting at responses
Allows student to make mistakes, but gives answers to all student questions
Allows mistakes, gives no answers to students, suggests a correct answer
Allows mistakes, no answers, no suggested correct answer, asks probing questions

38. Scaled Performance Item Example
Instructor Performance Demonstrating Knowledge of Student Assumptions
1. Instructor was not concerned with assumptions
2. Instructor asked questions about assumptions
3. Instructor asked questions about assumptions and asked students to project an outcome based on assumptions
4. Instructor asked questions, asked for projections, and asked how assumptions apply to other situations or asked probing questions

39. QUESTIONS ?

40. CT Performance Measures
What scale of assessment could be used for the instructor from “Men in Black” regarding student assumptions?

41. Example Item

43. Summary Defined Critical Thinking
Described how to recognize CT performance
Discusses goals of an evaluator who must assess instructor CT integration
Describes performances associated with CT
Practices developing a performance measure of CT

44. Final Questions or Comments