1. Candidate Evaluation

2. Candidate evaluation Instructors need evaluation skills for
Candidate evaluation
Instructors need evaluation skills for
Diagnostics
Skills
But also in competition
Outcome based assessment
Evaluating the students provides feedback on their understanding of the material and ability to perform a skill.
It is important to consider evaluation of skills within context of outcome based assessment - whether the student shows the ability to use and apply the knowledge.
In some skills, the student will need to perform the skill as per the manual – for example applying a cervical collar. In other skills there may be leeway in the application of the skill for treatments of dislocated shoulder, or in applying the body sling. The importance is to have the skill reflect the outcome desired - is the treatment doing the job in making the patient comfortable.
Property of the Canadian Ski Patrol System

3. Preparation Before evaluation Read, understand, assimilate forms
Review/discuss potential situations
On the day
Determine if this is a re-test
Review forms
Site setup
Ensure you have everything you need
Proper environment (noise, temperature)

4. Prepare patient Review scenario with patient
Prepare patient
Review scenario with patient
Ensure simulation and exam match
Ensure patient knows what to say and when
Position patient according to situation
Don’t make things up
Ensure patient looks at you regularly
Prompt when necessary
Property of the Canadian Ski Patrol System

5. Prepare candidate Write name on form Provide scenario instructions
Prepare candidate
Write name on form
Provide scenario instructions
Only provide information available from forms (don’t make things up)
Explain what is being tested
Skill, diagnostic, etc.
Ask if they have any questions
Verbalization with diagnostic is not required
You can prompt for clarification at the end if necessary
Property of the Canadian Ski Patrol System

6. Evaluation Give the candidate your full attention
Evaluation
Give the candidate your full attention
Only provide information candidate has obtained
Supervise patient
Try not to break the flow
Sequence is not mandatory in all situations
Give the candidate your full attention
-make sure you can see and hear
-make sure you are not distracted
Only provide information candidate has obtained
-don’t give information “for free”
Supervise patient
-ensure patient provides correct symptoms at the correct level
-don’t let the scenario get away from you adapt when you can’t fix
For example, when taking assessing pulse rate, if the candidate is only tells you the pulse rate, only give him the value for the pulse rate. Don’t provide the rhythm or strength.
If, somehow, the scenario is running away from the script, intervene to re-adjust. If this is not possible, adapt to the change and take this into account in your evaluation of the candidate, without penalizing him.
Some critical parts of diagnostics must be done in a specific order. Other parts are more flexible.
Property of the Canadian Ski Patrol System

7. Marking Stick to the exact text in each item “Starred” items
Marking
Stick to the exact text in each item
Literally, if it makes sense
Evaluate if candidate understood what was required
“Starred” items
Understand why they are starred
Ensure candidate gets a chance to perform the item
Prompt as required; don’t give mark away
If, for a reason other than a fault of his own, a candidate skips a point; use questioning techniques to give him an opportunity to nonetheless obtain the point.
Property of the Canadian Ski Patrol System

8.
Goals of prompting
Ensure candidate knows what he may not have verbalized
Go fishing for marks for a “deserving” candidate
When to prompt and when not to:
Patient and candidate
Prompting after the scenario usually works best
Often, candidates who know the subject matter have lower marks than they normally would due to the pressures and stress of being examined.
Consider candidate stress and account for simulation versus real life.
The idea here is to remove this pressure while ensuring that the results correctly reflect their level of knowledge.
Asking questions will help you determine if the candidate does indeed know the subject matter.
Property of the Canadian Ski Patrol System

9. Prompting the patient Timing of prompts
Prompting the patient
Timing of prompts
Do not give away the answer by your prompting
Ensure patient behaves according to scenario
Timing:
As required according to forms
When patient deviates from scenario
Make sure candidate is not interrupted
Ensure patient behaves according to scenario
Remind patient what to do or tell candidate what patient should have done
Property of the Canadian Ski Patrol System

10. Prompting the candidate
Prompting the candidate
Generally done at the end
Do not give away the answer by your question
Ask a few questions
“What specific treatment would be required in this specific situation?”
Best is not to interrupt the candidate during his performance. Wait until the end to ask questions.
The goal is to determine if the candidate knows the subject matter et whether he lost marks due to the stress of the examination.
Ask a few questions
For starred items go fishing a bit more
For other items, ask a question or two and move on if candidate does not readily answer
Property of the Canadian Ski Patrol System

11. Candidate stress Recognize symptoms Anxiousness, nervousness, sweating
Candidate stress
Recognize symptoms
Anxiousness, nervousness, sweating
“Blanking”, lack of focus, rambling
Retests:
Property of the Canadian Ski Patrol System

12.
Feedback
As soon as possible and practical, tell candidate whether they passed
Do not review every missed item
Do not get into an argument with the candidate
Discuss/question the patient aside/away from the candidate
Use the “sandwich/Oreo cookie” principle
Do not review every missed item
Concentrate on important points (for learning)
Oreo cookie:
Start with a positive comment
Ask the candidate to identify a weakness (guide candidate toward the main weakness you feel needs to be improved upon)
End with another positive comment
Property of the Canadian Ski Patrol System

13. After the evaluation
Give the candidate a chance to get obvious notes he knows he missed
Add some of the notes that a candidate mentions having missed (without prompting)
Comfort candidates who are failing; be positive about their chances of passing the eventual retest

14. Retests Case by case Consider starred item versus %
Learning opportunity

15. Don’t Be afraid to fail a candidate Try to fail a candidate
Make up stories/situations

16. Do Be consistent and fair Use evaluation as a training tool Do
Be consistent and fair
Use evaluation as a training tool
Consider the candidate is stressed
Use good judgment
Adapt to the flow of the scenario
Stay positive
Complete the full evaluation
As much as possible, never stop an evaluation in the middle; this is too demoralizing for the candidate.
Property of the Canadian Ski Patrol System

17. Finalize Follow-up with patient
Finalize
Follow-up with patient
Review, complete, and sign marking sheet
Hand in sheet
Return patient to simulation or next scenario
Reset station
Follow-up with Patient
Review items/marks you may have missed
Fill in patient-related portions of the form
Consider doing this away from the candidate
Review, complete, and sign marking sheet
Add up the marks, write score on first page
Indicate Pass/Retest/Fail
Property of the Canadian Ski Patrol System

18. Dispute resolution Minimize arguments Who to go to
Listen to the candidate
Consider discussing with another instructor
Be confident about your decision
Who to go to
Appeals go to the Education Officer
“Direct” the candidate to the Education Officer
Avoid continued confrontation