1. Approaches to Assessing Critical Thinking Kam-Por Kwan x6287 etkpkwan@ Patrick Lai x6294 etktlai@ EDU Lunchtime Seminar 12 May 2000

2. 2 Seminar series z12 May 2000: Approaches to Assessing Critical Thinking z18 May 2000:Overcoming the Challenges in Developing Students’ Critical Thinking z9 June 2000:Measuring Students’ Critical Thinking: Problems and Possible Solutions

3. 3 Outline of today’s session zOverview of approaches to assessing critical thinking (CT):  Examples  Relative advantages and disadvantages  Appropriateness for different purposes of CT assessment zApplicability to your context zPolyU Special Interest Group for Teaching and Assessing CT

4. 4 Purposes of CT assessments zProgramme/subject evaluation: evidence of success in promoting CT zAuthentic assessment:  evidence of effectiveness of CT instruction  motivating students to be better at CT zDiagnostic feedback on students’ CT level:  for designing instruction  for faciliating students’ growth & development zResearch into CT instructions zSelecting students for admission

5. 5 Definition of CT zEnnis (1993: 180): “Critical thinking is reasonable reflective thinking focused on deciding what to believe or do” zAPA Delpi Report (1990): “Critical thinking is the process of purposeful, self-regulatory judgement. This process gives reasoned consideration to evidence, contexts, conceptualisations, methods and criteria.”

6. 6 Deciding what to believe or do zJudge the credibility of sources zIdentify conclusions, reasons, and assumptions zJudge the quality of an argument: acceptability of its reasons, assumptions, and evidence zDevelop and defend an issue zAsk appropriate clarification questions zPlan experiments amd judge experimental designs zDefine terms in a way appropriate for the context zBe open-minded zTry to be well-informed zDraw conclusions when warranted, but with caution

7. 7 CT Dispositions and skills zTwo components of critical thinking (Facione, 1996):  Affective dispositions: the necessary attitudes and habits in approaching problems, questions, and issues for CT to take place  Cognitive skills: the mental ability needed for critical thinking

8. 8 Critical thinking dispositions Dispositions towards CT Inquisitiveness Open-mindedness Systematicity Analyticity CT Self confidence Cognitive maturity Truth-seeking

9. 9 Critical thinking skills Analysis Interpretation Inference Explanation Evaluation Self-regulation CT Skills Source: Facione (1996)

10. 10 Approaches to CT assessments zStudents’ self-reported skills / gains in CT zStudents’ performance on standardised published CT tests zInstructor-constructed CT tests zStudent interviews / protocol analysis zStudents’ performance on tasks requiring CT skills  performance assessment  rubric for assessing CT in students’ work

11. 11 Self-reported skills /gains in CT zStudents’ self reports on:  skills in CT  gains or progress in CT skills zExamples:  Items from CSEQ  Items from IDEA zRelatively easy and inexpensive to use zLess reliable and valid

12. 12 Published CT tests zStandardised tests with well-established psychometric properties zExamples:  The California Critical Thinking Test: College level (CCTST)  Cornell Critical Thinking Test, Level Z (1985)  Watson-Glaser Critical Thinking Appraisal (1980) zReliability and validity established, often with norms for comparison zExpensive to use, non subject-specific, implicit cultural and contextual assumptions

13. 13 Instructor-constructed CT tests zDeveloped by instructor zCan be tailored for specific subject, context, or purpose zRelatively costly to design (both in time and effort), reliability and validity need to be determined, difficult to compare across groups or contexts

14. 14 Interview or protocol analysis zStudents given a problem to solve in an interview  to find out ways person understands a concept or solves a problem zResearcher analyses students’ explanation, and infers from that to form the researcher’s description zResult consists of categories which “describe different ways of understanding a concept or solving a problem” zThese categories stand in a logical hierarchical relationship with one another and have implications for teaching

15. 15 Interview method zStudents given a problem to solve in an interview zThey are asked to think aloud and tell the interviewer their thoughts as they are going through zProbing/Prompted questions  You mentioned about X. How does this occur?  Do these symbols have any meaning to you?  Why did you give up your original hypothesis?

16. 16 Protocol / data analysis zVerbatim transcription of audio-taped interviews zTranscription printed and read through with a particular research question in mind zDescriptive words or phrases attached to short sections of the transcript as a potential category zSections of the transcript that could fit into this potential category labelled by sub-categories zContent of one excerpt compared and contrasted with the content of another similarly labelled excerpt zLabelled excerpt organised into flow charts.

17. 17 Performance assessments zStudents’ written and / oral descriptions analysed by the Holistic Critical Thinking Scoring Rubric (HCTSR) (Facione and Facione, 1994) zHCTSR measures  critical thinking skills to reach a judicious judgment which include analysis, interpretation, evaluation, inference, explanation  assesses dispositions to pursue reasons and evidence fair mindedly and open-mindedly to reach sound, objective decision zHCTSR consists of four levels - level 4 signifies a critical thinker

18. 18 Assessing CT under the 3-P model      

19. 19 CT Assessment for different purposes         

20. 20 SIG for teaching & assessing CT zSharing of resources and experiences in teaching and assessing CT through  e-newsletter  regular meetings zCollaboration in designing and evaluating CT instructions  trying out innovative methods for CT instruction  small-scale classroom research zContact persons  Patrick Lai: etktlai@polyu.edu.hk x6294  K. P. Kwan: etkpkwan@polyu.edu.hk x6287