Instructional Critical Thinking Assessment Item Development Thinking about Thinking: Methods to Assess Critical Thinking Instruction in Higher Education Demonstration of Instructional Critical Thinking Assessment Item Development Linda Lynch, PhD, Educational Researcher, United States Army

Objectives Define Critical Thinking Describe how to recognize CT performance Discuss goals of an evaluator who must assess instructor CT integration Describe performances associated with CT Practice developing a performance measure of CT

Why is Critical Thinking Important? Jean Piaget (1923) said that “the second most important goal of education is to form minds which can be critical, can verify, and not accept everything what is offered”, and…. Piaget, J (1932). The moral judgment of the child. London, UK: Kegan Paul.

Why is Critical Thinking Important? …the first goal was to “create men and women who are capable of doing new things, not simply repeating what other generations have done – men who are creative, inventive, and discoverers”.

What Does Critical Thinking Look Like?

?

What are Your Goals as a Program Evaluator Supporting Critical Thinking ? What did you want to learn? What are your goals? What would you like to take away from this demonstration?

Suggested Goals as a Program Evaluator Supporting Critical Thinking

Suggested Goals as a Program Evaluator Supporting Critical Thinking 1. Provide evidence that instructors are integrating Critical Thinking into instruction

Your Goals as a Program Evaluator Supporting Critical Thinking 1. Provide evidence that instructors are integrating Critical Thinking into instruction 2. YOUR ULTIMATE GOAL: Define performances that demonstrate quality CT instruction

How to Start Developing Performance Measures of CRITICAL THINKING INSTRUCTIONAL QUALITY ?

Evaluators ask these questions…

Does the instructor demonstrate …

Universal Intellectual Standards

Universal Intellectual Standards Accuracy: Clarity: Relevance: Logic: Sufficiency: Precision:

Universal Intellectual Standards Accuracy: Is that really true? Clarity: Could you elaborate? Relevance: How is that connected? Logic: Does that make sense? Sufficiency: Does that provide reason for all the outcomes? Precision: Could you give more details?

Universal Intellectual Standards Significance: Fairness: Breadth: Depth:

Universal Intellectual Standards Significance: Is this the most important problem to consider? Fairness: Do I have a vested interest in the outcome? Breadth: Should another opinion be considered? Depth: Am I taking into account the most significant factors?

Does the instructor demonstrate …

Essential Intellectual Traits

Essential Intellectual Traits Intellectual Humility: Sensitivity to bias prejudice and one’s own viewpoint Intellectual Autonomy: A commitment to analyzing and evaluating beliefs based on reason Fairmindedness: The need to treat all viewpoints alike Intellectual Courage: The need to face ideas need beliefs toward which there is strong negative emotion

Essential Intellectual Traits Confidence in Reason: The interests of humankind is served by encouraging the development of thinking skills Intellectual Perseverance: The need to use intellectual truths in spite of opposition Intellectual Empathy: The ability to understand the reasoning of others Intellectual Integrity: The need to be true to one’s own thinking

Does the instructor demonstrate …

Elements of Reasoning Purpose: All reasoning has a purpose that is stated clearly Questions: All reasoning is an attempt to answer a question or solve a problem Assumptions: All reasoning is based on Assumptions Points of View: All reasoning is accomplished from a point of view Information: All reasoning is based on Data and Information

Elements of Reasoning Inferences: All reasoning contains inferences and interpretations Concepts: All reasoning is expressed through and shaped by concepts and ideas Implications: All reasoning has implications and consequences

Critical Thinking Model Refer to Paul and Elder Critical Thinking Model Model: University of Louisville: Ideas to Action: Using Critical Thinking to Foster Student Learning and Community Engagement https://louisville.edu/ideastoaction/what/critical-thinking/paul-elder-framework

Review the Elements of Thought

Dr. Richard Paul & Dr. Linda Elder The Foundation and Center for Critical Thinking www.criticalthinking.org Paul, R Elder, Linda (2008) The Miniature Guide to Critical Thinking Concepts and Tools. Foundation for Critical Thinking Press

Through speech, support, prompting, or discussion. What is the instructor’s demonstration of critical thinking? Through speech, support, prompting, or discussion.

Group Directions

GROUP Directions Develop an assessment item that demonstrates the performance of an Intellectual Standard or Trait by an instructor 5 minutes

What Did You Choose to Measure? What did you Assess? Group answers

Instructor Performance Instructor Behavior What Should Be Measured or Assessed to Demonstrate Quality CT Instruction? Instructor Performance Instructor Behavior

What is an Example of an Instructor Performance Measure of Critical Thinking?

CT Performance Measures Assessment item responses suggest a level of performance

Example Performance Measure Does not allow students to make mistakes – hinting at responses Allows student to make mistakes, but gives answers to all student questions Allows mistakes, gives no answers to students, suggests a correct answer Allows mistakes, no answers, no suggested correct answer, asks probing questions

Scaled Performance Item Example Instructor Performance Demonstrating Knowledge of Student Assumptions 1. Instructor was not concerned with assumptions 2. Instructor asked questions about assumptions 3. Instructor asked questions about assumptions and asked students to project an outcome based on assumptions 4. Instructor asked questions, asked for projections, and asked how assumptions apply to other situations or asked probing questions

QUESTIONS ?

CT Performance Measures What scale of assessment could be used for the instructor from “Men in Black” regarding student assumptions?

Example Item

Summary Defined Critical Thinking Described how to recognize CT performance Discusses goals of an evaluator who must assess instructor CT integration Describes performances associated with CT Practices developing a performance measure of CT

Final Questions or Comments