Candidate Evaluation

Candidate evaluation Instructors need evaluation skills for 2008-09-14 Candidate evaluation Instructors need evaluation skills for Diagnostics Skills But also in competition Outcome based assessment Evaluating the students provides feedback on their understanding of the material and ability to perform a skill. It is important to consider evaluation of skills within context of outcome based assessment - whether the student shows the ability to use and apply the knowledge. In some skills, the student will need to perform the skill as per the manual – for example applying a cervical collar. In other skills there may be leeway in the application of the skill for treatments of dislocated shoulder, or in applying the body sling. The importance is to have the skill reflect the outcome desired - is the treatment doing the job in making the patient comfortable. Property of the Canadian Ski Patrol System

Preparation Before evaluation Read, understand, assimilate forms Review/discuss potential situations On the day Determine if this is a re-test Review forms Site setup Ensure you have everything you need Proper environment (noise, temperature)

Prepare patient Review scenario with patient 2008-09-14 Prepare patient Review scenario with patient Ensure simulation and exam match Ensure patient knows what to say and when Position patient according to situation Don’t make things up Ensure patient looks at you regularly Prompt when necessary Property of the Canadian Ski Patrol System

Prepare candidate Write name on form Provide scenario instructions 2008-09-14 Prepare candidate Write name on form Provide scenario instructions Only provide information available from forms (don’t make things up) Explain what is being tested Skill, diagnostic, etc. Ask if they have any questions Verbalization with diagnostic is not required You can prompt for clarification at the end if necessary Property of the Canadian Ski Patrol System

Evaluation Give the candidate your full attention 2008-09-14 Evaluation Give the candidate your full attention Only provide information candidate has obtained Supervise patient Try not to break the flow Sequence is not mandatory in all situations Give the candidate your full attention -make sure you can see and hear -make sure you are not distracted Only provide information candidate has obtained -don’t give information “for free” Supervise patient -ensure patient provides correct symptoms at the correct level -don’t let the scenario get away from you adapt when you can’t fix For example, when taking assessing pulse rate, if the candidate is only tells you the pulse rate, only give him the value for the pulse rate. Don’t provide the rhythm or strength. If, somehow, the scenario is running away from the script, intervene to re-adjust. If this is not possible, adapt to the change and take this into account in your evaluation of the candidate, without penalizing him. Some critical parts of diagnostics must be done in a specific order. Other parts are more flexible. Property of the Canadian Ski Patrol System

Marking Stick to the exact text in each item “Starred” items 2008-09-14 Marking Stick to the exact text in each item Literally, if it makes sense Evaluate if candidate understood what was required “Starred” items Understand why they are starred Ensure candidate gets a chance to perform the item Prompt as required; don’t give mark away If, for a reason other than a fault of his own, a candidate skips a point; use questioning techniques to give him an opportunity to nonetheless obtain the point. Property of the Canadian Ski Patrol System

2008-09-14 Goals of prompting Ensure candidate knows what he may not have verbalized Go fishing for marks for a “deserving” candidate When to prompt and when not to: Patient and candidate Prompting after the scenario usually works best Often, candidates who know the subject matter have lower marks than they normally would due to the pressures and stress of being examined. Consider candidate stress and account for simulation versus real life. The idea here is to remove this pressure while ensuring that the results correctly reflect their level of knowledge. Asking questions will help you determine if the candidate does indeed know the subject matter. Property of the Canadian Ski Patrol System

Prompting the patient Timing of prompts 2008-09-14 Prompting the patient Timing of prompts Do not give away the answer by your prompting Ensure patient behaves according to scenario Timing: As required according to forms When patient deviates from scenario Make sure candidate is not interrupted Ensure patient behaves according to scenario Remind patient what to do or tell candidate what patient should have done Property of the Canadian Ski Patrol System

Prompting the candidate 2008-09-14 Prompting the candidate Generally done at the end Do not give away the answer by your question Ask a few questions “What specific treatment would be required in this specific situation?” Best is not to interrupt the candidate during his performance. Wait until the end to ask questions. The goal is to determine if the candidate knows the subject matter et whether he lost marks due to the stress of the examination. Ask a few questions For starred items go fishing a bit more For other items, ask a question or two and move on if candidate does not readily answer Property of the Canadian Ski Patrol System

Candidate stress Recognize symptoms Anxiousness, nervousness, sweating 2008-09-14 Candidate stress Recognize symptoms Anxiousness, nervousness, sweating “Blanking”, lack of focus, rambling Retests: Property of the Canadian Ski Patrol System

2008-09-14 Feedback As soon as possible and practical, tell candidate whether they passed Do not review every missed item Do not get into an argument with the candidate Discuss/question the patient aside/away from the candidate Use the “sandwich/Oreo cookie” principle Do not review every missed item Concentrate on important points (for learning) Oreo cookie: Start with a positive comment Ask the candidate to identify a weakness (guide candidate toward the main weakness you feel needs to be improved upon) End with another positive comment Property of the Canadian Ski Patrol System

After the evaluation Give the candidate a chance to get obvious notes he knows he missed Add some of the notes that a candidate mentions having missed (without prompting) Comfort candidates who are failing; be positive about their chances of passing the eventual retest

Retests Case by case Consider starred item versus % Learning opportunity

Don’t Be afraid to fail a candidate Try to fail a candidate Make up stories/situations

Do Be consistent and fair Use evaluation as a training tool 2008-09-14 Do Be consistent and fair Use evaluation as a training tool Consider the candidate is stressed Use good judgment Adapt to the flow of the scenario Stay positive Complete the full evaluation As much as possible, never stop an evaluation in the middle; this is too demoralizing for the candidate. Property of the Canadian Ski Patrol System

Finalize Follow-up with patient 2008-09-14 Finalize Follow-up with patient Review, complete, and sign marking sheet Hand in sheet Return patient to simulation or next scenario Reset station Follow-up with Patient Review items/marks you may have missed Fill in patient-related portions of the form Consider doing this away from the candidate Review, complete, and sign marking sheet Add up the marks, write score on first page Indicate Pass/Retest/Fail Property of the Canadian Ski Patrol System

Dispute resolution Minimize arguments Who to go to Listen to the candidate Consider discussing with another instructor Be confident about your decision Who to go to Appeals go to the Education Officer “Direct” the candidate to the Education Officer Avoid continued confrontation