HCI – NYGH IP ED SEMINAR th Jan 2011 Diagnostic Teaching through Identification of Scientific Misconceptions Using Just-In-Time Teaching (JITT) and Certainty Of Response Index (CRI)

Overview Scientific misconceptions that students possessed are known to hamper the proper acceptance and integration of new knowledge or skills. These misconceptions are binding cognitive structures that do not match what are known to be scientifically correct. This presentation will describe the process of diagnosing student abilities and needs quantitatively in order to prescribe requisite learning activities through the use of Just-in-Time Teaching (JiTT) and Certainty of Response Index (CRI).

What is Just-in-Time Teaching (JiTT) Just-in-Time Teaching (JiTT) is a teaching and learning strategy based on the interaction between web-based study assignments and an active learner classroom. Students respond electronically to carefully constructed web- based assignments which are due shortly before class, and the instructor studys the student submissions "just-in-time" to adjust the classroom lesson to suit the students' needs. The heart of JiTT is the "feedback loop" formed by the students' outside-of-class preparation that fundamentally affects what happens during the subsequent in-class time together.

What is Certainty of Response Index (CRI) The Certainty of Response Index (CRI) is used in social sciences, particularly in surveys, where a respondent is requested to provide the degree of certainty he has in his own ability to select and utilize well-established knowledge, concepts or laws to arrive at the answer. The CRI is based on some scale. For example, the six-point scale (0–5) in which 0 implies no knowledge (total guess) of methods or laws required for answering a particular question while 5 indicates complete confidence in the knowledge of the principles and laws required to arrive at the selected answer. The methodology presented here is based on the paper written on Misconceptions and Certainty of Response Index (CRI) by Hasan, Bagayoko and Kelly (1999).

Low CRI (<2.5)High CRI (>=2.5) Correct AnswerCorrect answer and low CRI – Lack of knowledge (lucky guess) Correct answer and high CRI – Knowledge of correct concepts Wrong AnswerWrong answer and low CRI – Lack of knowledge Wrong answer and high CRI - Misconception Decision matrix for an individual student and for a given question.

Low CRI (<2.5)High CRI (>=2.5) Correct AnswerCorrect answer and low average CRI – Lack of knowledge (lucky guess) Correct answer and high average CRI – Knowledge of correct concepts Wrong AnswerWrong answer and low average CRI – Lack of knowledge Wrong answer and high average CRI - Misconception Decision matrix for a group of students (class) and for a given question.

HCI Wikispaces – Instructional Platform The first step was to create a web-based instructional platform using Wikispaces that incorporates lesson material for downloading and learning by students as well as embedding videos for students to aid understanding. Students were also tasked to answer 10 multiple choice questions. In addition, the students were required to choose the level of confidence that they had answered correctly for every question. For every question answered, the students had to choose the certainty of their answer using: (0 – Totally guessed answer, 1 – Almost a guess, 2 – Not sure, 3 – Sure, 4 – Almost certain, 5 – Certain.)

Capturing Students’ Responses The results from the answer and CRI were captured to Excel worksheet and processed using Google Forms.

Results

An Example If we turn our attention to Q6 for the 3 classes (image 5, 6 and 7), Q6 had the lowest number of students who got it correct and its CRI was quite high (about 3.0). This indicated that students who answered this question wrongly were confident that they would get it right. Thus, the students had misconceptions and not a lack of knowledge and the teacher would be able to diagnose which area of the topic that students in general possessed misconceptions and thus would be able to address them effectively.

Survey A survey was administered to the students (n=49) using Likert scale (image 9) and there are a total of 5 strands for this survey (see questions from survey results). Each strand is asked in both positively and negatively in order to test the validity of the response. The survey results are encouraging as students in general posted positive responses

Survey Results In order to quantify the assembly of these inter-related strands measuring the underlying construct (Students’ perceptions towards diagnostic teaching), I used the index of reliability – Cronbach Alpha. This index gauges the internal consistency or average correlation of items in a survey instrument to gauge its reliability. Cronbach Alpha was calculated to be 0.79 (n=49). Nunnaly (1978) has indicated 0.70 and above to be an acceptable reliability coefficient for Cronbach Alpha.

Qualitative Survey In addition, I present several qualitative proofs of the efficacy of diagnostic teaching below: 1) Charles Low (3P2) - Diagnostic Teaching is an innovative new way of teaching that brings a new dimension of learning. 2) Damon Tan (3O2) - I would think that the whole idea of diagnostic teaching is a very good idea, as it focuses on our weakness. However, this means that there is a large amount of independent studies at home, taking up time, and there is lesser teaching in class in terms of general areas. I would suggest that diagnostic teaching be continued, however, a balance should be struck so that there is an equal amount of time spent on teaching "standard" curriculum. Nonetheless, I would like to thank you for introducing this great way to learn. It has greatly motivated me to learn physics :) 3) Zhao Jinqing (3P1) - I wish for more diagnostic teaching in the future.

Conclusion The identification of misconceptions using diagnostic teaching utilizing Just-in-Time Teaching (JiTT) and Certainty of Response Index (CRI) can be introduced and used by other subjects and research has shown that it can easily be inducted and carried out by all educational fields. From the results, the various scientific misconceptions could be cataloged and used for future teaching and learning as it is based on empirical studies and not anecdotal evidence.

Reference Hasan, S., Bagayoko, D., & Kelly, E.L. (1999). Misconceptions and the Certainty of Response Index (CRI), Phys. Educ. 34, 294. Nunnaly, J. (1978). Psychometric Theory. McGraw-Hill, new York.

Thank You for your kind attention Q & A