Examination Student Examiner Student 8-Nov-18 Dr. Jatin Bhatt

Structured Oral Viva Dr. Jatin G. Bhatt Professor & Head Department of Surgery PDUMC, Rajkot. 8-Nov-18 Dr. Jatin Bhatt

Objective Abilities that can be tested —Classification Structure —Merits and Demerits Strategies to improve 8-Nov-18 Dr. Jatin Bhatt

Objectives At the end of session, the participant will be able to implement strategies to improve the oral examination in his/her department. Write down one strategy for your dept…….. 8-Nov-18 Dr. Jatin Bhatt

What is an oral assessment? Direct means of assessing students’ learning outcomes by questioning them — ‘Oral assessment’ includes any assessment of student learning that is conducted, wholly or in part, by word of mouth 8-Nov-18 Dr. Jatin Bhatt

Purpose Tests mainly Cognitive domain of the student, but can include affective or communication Memory(depth of knowledge) Understanding of the subject Reasoning and problem solving Communication skills Ability to perform under stress Professional competence 8-Nov-18 Dr. Jatin Bhatt

Types of oral assessments 1. Oral assessment after a direct observation assessment   — part of a de-briefing session after a practical has been observed. —time duration:3-5 minutes. usually no formal structure ask questions as examiner foresee however, assessors may plan some general questions for all. 8-Nov-18 Dr. Jatin Bhatt

Types of oral assessments 2. Oral in the form of a viva voce — Latin name for oral examination —often given for a university examination with spoken questions and answers Usually PG level, after the submission of the thesis Vivas are traditionally conducted by an external and an internal examiner no set time limit 8-Nov-18 Dr. Jatin Bhatt

Types of oral assessments 3. Oral/Aural in a language setting Oral is a direct speaking test assessing a student's level of speaking proficiency. Aural is a listening test assessing a student's level of hearing proficiency. 8-Nov-18 Dr. Jatin Bhatt

Types of oral assessments 4. Miscellaneous: — Table Viva — Long case — Short Case 8-Nov-18 Dr. Jatin Bhatt

Declaration All the characters in the role plays are fictitious. Resemblance if any to the living or dead is purely coincidental. Role Play 8-Nov-18 Dr. Jatin Bhatt

Role Play Examination: Final M.B.,B.S. University Examination Subject: Surgery Examiner: Dr. Jatin Bhatt Dr. G. U. Kavithia Student: Dr. Pratik Dr. Arohi Write down demerits 8-Nov-18 Dr. Jatin Bhatt

DEMERITS 8-Nov-18 Dr. Jatin Bhatt

Demerits – Extraneous Factors Very subjective Luck factor Bias and impressions : “The halo effect "gender, accent, outward appearance, confidence Language barrier Undue advantage of fluency and style Unnecessary negative feedback 8-Nov-18 Dr. Jatin Bhatt

Demerits Lacks standardization Lacks objectivity and reliability Lacks structure and organization Impatient Examiner : Stressed examinee Examiner sympathy : number of failures Lack of interest / other preoccupation of the examiner Disproportionate time and resources are spent 8-Nov-18 Dr. Jatin Bhatt

Demerits — Few trained examiners available — Novelty :   — form of oral assessment being used may be unfamiliar to the student   — Articulateness Vs knowledge :   —Examiners can mistake a student’s articulateness for knowledge 8-Nov-18 Dr. Jatin Bhatt

Structure of an oral assessment Depends on type of oral assessment, but in general, the followings are used. Allow the student to start His spontaneous account can reveal far more than simply responses to the questions. Questions such as: How do you think you did? What is the probable diagnosis according to you? 8-Nov-18 Dr. Jatin Bhatt

Structure of an oral assessment Probing questions - to initiate and engage. Questions such as: How did you know that? What method did you use to arrive with that conclusion? 8-Nov-18 Dr. Jatin Bhatt

Structure of an oral assessment Prompting questions – give hints Questions such as: Remember the experiment on xyz? What do you think this relates to? Structures in right hypochondrium (related to the liver) 8-Nov-18 Dr. Jatin Bhatt

Structure of an oral assessment Challenging questions – to assess deep understanding - higher level of Blooms taxonomy. Questions such as: Can you justify why your method is more efficient than Prof. Einstein’s? Why should I consider your diagnosis? 8-Nov-18 Dr. Jatin Bhatt

Validity Assessment is valid when it allows students to fully demonstrate their knowledge, skills and values in relation to the course they are studying. Several types of validity that can throw light on the quality of oral assessment. 8-Nov-18 Dr. Jatin Bhatt

Reliability When a student has completed an oral assessment, can we be confident that he or she would do equally well if they were asked other questions? Given a different scenario to respond to? Assessed by another colleague? Assessed at another time? 8-Nov-18 Dr. Jatin Bhatt

Role Play Ideal Method Write down merits 8-Nov-18 Dr. Jatin Bhatt

MERITS 8-Nov-18 Dr. Jatin Bhatt

Merits Direct personal contact with student Direct personal communication Tests all levels : cognitive, psychomotor, affective Assess problem solving and reasoning Assess professionalism and ethics Probe depth of knowledge Flexibility to move from one area to another Feedback on curriculum Tailor question as per student 8-Nov-18 Dr. Jatin Bhatt

Merits Analytical skills and judgment Samples a wide range of knowledge Fast thinking and clear expression Borderline cases To award distinctions Assesses personality and aptitude Tests interactive skills Accommodate circumstantial factors 8-Nov-18 Dr. Jatin Bhatt

Merits Opportunity to defend statement Less scope of cheating by the examinee Individual and group problems in teaching and learning can be identified Valuable immediate feedback for students, teachers and educational process 8-Nov-18 Dr. Jatin Bhatt

Common beliefs about viva Examined at the beginning or end of session Fluency fetches more marks Female candidates favored Last candidates are favored Some tables are “toxic” 8-Nov-18 Dr. Jatin Bhatt

Some facts about viva Lack of reliability Inter-examiner reliability very low Distinction marking trends seen: By examiner Examining team By hour By day General tendency towards leniency Errors of contrast : marks depend upon who gave viva before you 8-Nov-18 Dr. Jatin Bhatt

Viva – some facts LACK OF OBJECTIVITY Arbitrary marking Irrelevant and trivial questions asked Lack of clarity in questioning Variable level of difficulty of questions No guidelines given to examiner No agreement between examiners as to what is the correct answer : for that model answer should be structured Error of logic : Examiner is not up to date with advances Examiner expects an answer which he thinks is correct – unwilling to listen to any alternative 8-Nov-18 Dr. Jatin Bhatt

Discussion on strategy to improve Oral Examination in Dept. 8-Nov-18 Dr. Jatin Bhatt

Conduct of examinations Meeting of examiners to be called in advance… tell them Stick to time schedule Proceed from easy to difficult questions Test all levels of knowledge in all domains Maintain proper sequence of questioning and deviate when necessary Graded questions on colored cards Mark independently 8-Nov-18 Dr. Jatin Bhatt

Conduct of examinations Make students comfortable, relieve them of anxiety Appoint sufficient number of examiners At least two examiners to examine candidate simultaneously Many short sessions preferable to one long session Plan to start a new topic every 5 minute ( At least six topics in a half hour oral ) 8-Nov-18 Dr. Jatin Bhatt

Techniques during viva Spend a few moments putting a candidate at ease Introduce each topic and define its area. Go to the core of the question quickly using short questions and avoiding detailed scenarios Avoid factual questions and unmarkable questions If you plan to use props( letter, pictures, ECGs ) make sure that their function is clear and that they enhance the testing process and do not waste time 8-Nov-18 Dr. Jatin Bhatt

Techniques during viva Plan tactics for difficult candidates – slow candidates, garrulous(talkative) candidate Poor candidate may need to be encouraged Use non verbal encouragement The more topics covered in an oral examination the better When the bell goes, let the candidate finish his or her sentence before closing the examination 8-Nov-18 Dr. Jatin Bhatt

Higher order questioning OPEN QUESTIONING Can elicit many possible answers from candidate Can evaluate better/ extract more information Close question: what is normal blood sugar level ? Open question : What is importance of knowing a patients blood sugar ? 8-Nov-18 Dr. Jatin Bhatt

Probing questions To elicit precise answers from a candidate when tentative or vague in his answers Initial Q : In a case of Thyrotoxicosis, what test would you get done ? Answer : Thyroid function tests Probing Q : Tell me, what specific tests do you need in this case ? 8-Nov-18 Dr. Jatin Bhatt

Hypothetical Question If a patient of acute abdomen comes in emergency with x-ray showing free gas under domes of Diaphragm what will you do? 8-Nov-18 Dr. Jatin Bhatt

Challenge Question In a case of Lipoma…. Why this is not a Sebaceous cyst? In case of palpable Spleen…. Why not a Kidney? 8-Nov-18 Dr. Jatin Bhatt

Interactive style of questioning Uses all types described above Subsequent questions are based on the candidate’s replies This interactive style is Friendly Probing Test the depth of knowledge Experienced examiners adopt this style to good effect. 8-Nov-18 Dr. Jatin Bhatt

Interactive Style What is your Diagnosis? Direct or Indirect? Why? Inguinal hernia Direct or Indirect? Direct Why? Deep ring occlusion test –ve Demonstrate it?.............. ???????? 8-Nov-18 Dr. Jatin Bhatt

Structured viva Question should be: Questions should be: Clear Short Precise Unequivocal Questions should be: Cover all subtasks In proper sequence Test all levels of knowledge in different domains Give standard set of questions to all examiners Prepare standard marking system 8-Nov-18 Dr. Jatin Bhatt

Structured viva Other aspects : Prepare rating scale Fluency, confidence, expressive power Prepare rating scale Inadequate Adequate Satisfactory Take feedback from peers, students, teachers, colleagues, superiors- critically evaluate performance Recycle viva voce project by planning, replanning and revaluation. 8-Nov-18 Dr. Jatin Bhatt

Examiner should Be helpful Provide clues Reward 8-Nov-18 Dr. Jatin Bhatt

Do not…. Persist with a question Interrupt the candidate Teach Contradict Harass the student 8-Nov-18 Dr. Jatin Bhatt

To create accountability on part of the examiners Record the proceedings Written Audio video 8-Nov-18 Dr. Jatin Bhatt

Conclusion The Oral Examination is a “rite of passage” and forms a common bond between examiners. Giving up such a tradition is not easy and it will continue to have a place in assessment system. However need is to develop traditional oral examination into a “best practice oral” format. 8-Nov-18 Dr. Jatin Bhatt

Thanks 8-Nov-18 Dr. Jatin Bhatt