Structured Oral Viva Dr Gauravi A. Dhruva Prof. & Head Department of Pathology P. D. U. Medical College, Rajkot

Objective Abilities that can be tested —Classification Structure —Merits and Demerits Strategies to improve

What is an oral assessment? Direct means of assessing students’ learning outcomes by questioning them — ‘Oral assessment’ includes any assessment of student learning that is conducted, wholly or in part, by word of mouth

Purpose Tests mainly Cognitive domain of the student, but can include affective or communication Memory(depth of knowledge) Understanding of the subject Reasoning and problem solving Communication skills Ability to perform under stress Professional competence

Types of oral assessments 1. Oral assessment after a direct observation assessment   — part of a de-briefing session after a practical has been observed. —time duration:3-5 minutes. usually no formal structure ask questions as examiner foresee however, assessors may plan some general questions for all.

Types of oral assessments 2. Oral in the form of a viva voce — Latin name for oral examination —often given for a university examination with spoken questions and answers Usually PG level, after the submission of the thesis Vivas are traditionally conducted by an external and an internal examiner no set time limit

Types of oral assessments 3. Oral/Aural in a language setting Oral is a direct speaking test assessing a student's level of speaking proficiency. Aural is a listening test assessing a student's level of hearing proficiency.

Types of oral assessments 4. Miscellaneous: — Table Viva — Long case — Short Case

Structure of an oral assessment Depends on type of oral assessment, but in general, the followings are used. Allow the student to start His spontaneous account can reveal far more than simply responses to the questions. Questions such as: How do you think you did? What is the probable diagnosis according to you?

Structure of an oral assessment Probing questions - to initiate and engage. Questions such as: How did you know that? What method did you use to arrive with that conclusion?

Structure of an oral assessment Prompting questions – give hints Questions such as: Remember the experiment on xyz? What do you think this relates to? Structures in right hypochondrium (related to the liver)

Structure of an oral assessment Challenging questions – to assess deep understanding - higher level of Blooms taxonomy. Questions such as: Can you justify why your method is more efficient than Prof. Einstein’s? Why should I consider your diagnosis?

Validity Assessment is valid when it allows students to fully demonstrate their knowledge, skills and values in relation to the course they are studying. Several types of validity that can throw light on the quality of oral assessment.

Reliability When a student has completed an oral assessment, can we be confident that he or she would do equally well if they were asked other questions? Given a different scenario to respond to? Assessed by another colleague? Assessed at another time?

MERITS

Merits Direct personal communication Assesses personality and aptitude Circumstantial factors Flexible---Weak strong Direct personal contact with student Tests interactive skills Tests all levels : cognitive, psychomotor, affective

Merits Analytical skills and judgment Samples a wide range of knowledge Effective communication skills Fast thinking and clear expression Borderline cases To award distinctintions

Merits Opportunity to defend statement if required Understanding of subject – thoroughly tested Less scope of cheating by the examinee Individual and group problems in teaching and learning can be identified Valuable immediate feedback for students, teachers and educational process

DEMERITS

Demerits – Extraneous Factors Very subjective Luck factor Bias and impressions : “The halo effect "gender, accent, outward appearance, confidence Impatient Examiner : Stressed examinee Examiner sympathy : number of failures Lack of interest / other preoccupation of the examiner Unnecessary negative feedback

Demerits Lacks standardization Lacks objectivity and reliability Lacks structure and organization Subjective Language barrier Undue advantage of fluency and style Disproportionate time and resources are spent

Demerits — Few trained examiners available — Novelty :   — form of oral assessment being used may be unfamiliar to the student   — Articulateness Vs knowledge :   —Examiners can mistake a student’s articulateness for knowledge 12/04/2012

Examiners in viva Most examiners use this to only test recall Only reel off facts… no attempt to test problem solving ability Rarely any attempt to find out depth of knowledge Duration of viva is generally too short

For the student ?? Importance of viva voce Convey the fullest extent of understanding to the examiner Behave normally … do not try to impress Be composed … no medicines please !!

For the student Do not repeat questions Say no… instead of beating around the bush or trying to bluff ! Avoid vagueness Enter and leave graciously… both first and last impression are important

Purpose Preclinical To cover areas not tested in practical examination or for testing additional knowledge Para-clinical To test additional skills like identifying slides, spots and specimens Clinical Emphasis is on X-rays, ECGs, identification of instruments and specimens

Common beliefs about viva Marks obtained depend on whether candidate is examined at the beginning or end of session. Fluency fetches more marks Female candidates favored Last candidates are favored Some tables are “toxic”

Some facts about viva Lack of reliability Inter-examiner reliability very low Distinction marking trends seen: By examiner Examining team By hour By day General tendency towards leniency Errors of contrast : marks depend upon who gave viva before you

Some facts about viva Marks awarded by examiner correlated positively and significantly with Number of words used by student Speaking time of the student during the exam Marks awarded correlated negatively and significantly with Number of words used by examiner Percentage of time used by examiner No significant correlation with same student’s marks in an objective test on same content material ( Evans et al )

Viva – some facts LACK OF OBJECTIVITY Arbitrary marking Irrelevant and trivial questions asked Lack of clarity in questioning Variable level of difficulty of questions No guidelines given to examiner No agreement between examiners as to what is the correct answer : for that model answer should be structured Error of logic : Examiner is not up to date with advances Examiner expects an answer which he thinks is correct – unwilling to listen to any alternative

Conduct of examinations Meeting of examiners to be called in advance… tell them Stick to time schedule Proceed from easy to difficult questions Test all levels of knowledge in all domains Maintain proper sequence of questioning and deviate when necessary Graded questions on colored cards Mark independently

Conduct of examinations Make students comfortable, relieve them of anxiety Appoint sufficient number of examiners At least two examiners to examine candidate simultaneously Many short sessions preferable to one long session Plan to start a new topic every minute ( At least six topics in a half hour oral )

Techniques during viva Spend a few moments putting a candidate at ease Introduce each topic and define its area. Go to the core of the question quickly using short questions and avoiding detailed scenarios Avoid factual questions and unmarkable questions If you plan to use props( letter, pictures, ECGs ) make sure that their function is clear and that they enhance the testing process and do not waste time

Techniques during viva Plan tactics for difficult candidates – slow candidates, garrulous candidate Poor candidate may need to be encouraged Use non verbal encouragement The more topics covered in an oral examination the better When the bell goes, let the candidate finish his or her sentence before closing the examination

Higher order questioning OPEN QUESTIONING Can elicit many possible answers from candidate Can evaluate better/ extract more information Close question: what is normal blood sugar level ? Open question : What is importance of knowing a patients blood sugar ?

Probing questions Useful to elicit precise answers from a candidate when tentative or vague in his answers May be of closed or open type and depend on the answer to the earlier query. Initial Q : In this respiratory case, what test would you get done ? Answer : Lung function tests, madam--- Probing Q : Tell me, what specific tests do you need in this case ?

Hypothetical Question An open question relating to a hypothetical situation Used for oral simulations, or to test the problem solving ability What would you do if ----- (describe the hypothetical situation) Shall we now do a role play ? Assume that a am obese type 2 diabetic and you are my doctor. Give me diet and general advice

Challenge Question An open question that challenges the candidate to provide further proof or details to justify his / her answer In this case, what are the odd points which do not fit in? If I ask you to argue against what you have just now said, what would you say ? Explain the minimum criteria you need to justify your diagnosis.

Interactive style of questioning Uses all types described above but after the initial query, subsequent questions are based on the candidate’s replies. This interactive style is friendly, probing and test the depth of knowledge of the candidate. Experienced examiners adopt this style to good effect.

Improving oral examinations Improving validity, reliability and objectivity OBJECTIVE STRUCTURED APPROACH Requires advance planning Prepare lists of tasks and ability to be tested by oral examination According to core curriculum and additional areas Cognitive, affective and communication areas Prepare concrete examples Formulate questions Reach an agreement-- ?? Constitutes a correct answer

Structured viva Question should be : Clear Short Precise Unequivocal Questions should Cover all subtasks In proper sequence Test all levels of knowledge in different domains Give standard set of questions to all examiners Prepare standard marking system

Structured viva Other aspects : Fluency, confidence, expressive power Prepare rating scale with agreed examples and agreed criteria of correct answers, perhaps, with a classification in to Inadequate Adequate Satisfactory After viva, take feedback from peers, students, teachers, colleagues, superiors- critically evaluate performance Recycle viva voce project by planning, replanning and revaluation.