1. Assessing the practical/clinical skills of students: Presented at the RITAL Conference 2010 By: Florence Davidson, Radiography Lecturer, CPUT, GSH campus & Curriculum Officer for Radiography 7 th December 2010

2. Type of assessment Simulation Simulation OSCE OSCE Clinical (practical, in the workplace) Clinical (practical, in the workplace)

3. Contents: Presentation: (15 minutes) Presentation: (15 minutes) Defining “clinical competence” Defining “clinical competence” Types of instruments used for assessing competence Types of instruments used for assessing competence Drafting criteria for rating a clinical skill Drafting criteria for rating a clinical skill Aspects of defining “performance” Aspects of defining “performance” Additional considerations Additional considerations Round table discussion: (30 minutes) Round table discussion: (30 minutes)

4. A model of competence Fitness for purpose: focuses on the immediate requirements of the workplace Fitness for award: focuses on the educational preparation of the graduate Fitness for practice: focuses on the legal aspects of the individual practitioner’s capabilities A model of competence (Williams and Berry, 1999) CPUT GRADUATE

5. Defining clinical competence The Dreyfus and Dreyfus model of skill acquisition (adapted from described Benner (2001.) KEY: Novice: 1 st time in clinical, no experience. Advanced beginner: some clinical exposure, but unable to adapt to different situations. Competent performer: typically at the end of training/ mentoring, able to cope, ready registration. Proficient performer: has experience to modify their response. Expert: has an intuitive grasp, can “hone in” on the problem.

6. Observable Practice Potential practice Professional field OP: what the HCP is observed doing in practice * PP: what the HCP does not ordinarily do (?specialties, ? extended scope), but there may be reasonable expectation PF: holistic practice encompassing all the above as well as key competencies KEY: Adapted from Melnick et al (2002 ) Practice domains and assessment

7. “Technical ability” What are we assessing? Motor skills Previous experiences Cognitive elements Adaptability to new and changing situations Ability to work with peers and other Health Care Professionals Procedural checklists Global Rating scales

8. Instruments for assessing “technical” skills: 1. Structured Technical Skills Assessment Form (STSAF)

9. Instruments for assessing “technical” skills: 2. Global Rating Scales (GRS): Respect for tissue Often used unnecessary force on tissue or caused damage by inappropriate use of instruments Careful handling of tissue but occasionally caused inadvertent damage Consistently handled tissues appropriately, with minimal damage Time and motion Many unnecessary movesEfficient time and motion, but some unnecessary moves Economy of movement and maximum efficiency Instrument handling Repeatedly makes tentative or awkward moves with instruments Competent use of instruments, although occasionally appeared stiff and awkward Fluid moves with instruments and no awkwardness Knowledge of instruments Frequently asked for the wrong instrument or used an inappropriate instrument Knew the names of most instruments and used appropriate instrument for the task Obviously familiar with the instruments required and their names Use of assistants Flow of operation and forward planning Knowledge of specific procedure (Krishna Moorthy, Yaron Munz, Sudip K Sarker, Ara Darz:2003)i Variable 1 2 3 4 5

10. Refer to article: Objective assessment of technical skills in surgery Krishna Moorthy,

11. Activity: 1. Read the short excerpt entitled: ‘Paramedics’. 2. On the table provided, complete the following: Level of study Type of assess ment Reason

12. An example of a clinical assessment tool

13. Context 1 st Year Radiography students: Clinical Radiographic Practice 1 1 st Year Radiography students: Clinical Radiographic Practice 1  Students will have had 5 weeks experience in the clinical department (workplace- hospital);  Students will have covered the theoretical aspects in class;  Students will have attended practical demonstration (shown how);  Students will have “role played” on each other (do under controlled circumstances);  Students are given opportunity to comment on each other’s performance (peer evaluation)  Students will then perform the task under actual conditions (in reality) under supervision  Students will have had access to the assessment criteria before the assessment date (transparency)

14. Instructions to students 1.Please arrive in the x-ray department 30 minutes before your scheduled assessment time. 2.Find a suitable walking patient who requires the routine chest views. 3. Correctly identify the patient, introduce yourself to them as the student radiographer and ask if they would mind if you x- rayed them while the examiner marked you. 4.Introduce the patient to the examiner. 5. Proceed with the examination as per the criteria discussed in class.

15. Assessment criteria OBJECTIVE: Given a patient and the necessary radiographic equipment, the student must be able to perform a routine radiographic examination of the chest. OBJECTIVE: Given a patient and the necessary radiographic equipment, the student must be able to perform a routine radiographic examination of the chest. KEY PERFORMANCE AREAS: KEY PERFORMANCE AREAS:  Preparation for the examination  Patient care and communication  Production of a diagnostic quality radiograph of the routine projections

16. KEY PERFORMANCE AREAS: 1: PREPARATION FOR THE EXAMINATION:Comment 1.Evaluate the request form for procedure and patient information 2. Check previous radiographs (where appropriate) 3. Prepare physical facilities by providing clean, orderly work area 4. Obtain necessary equipment (cassettes, gonad shielding, gown, dripstand etc.) 5.Set the average exposure on the control panel for the appropriate examination 6. Select and place the cassette appropriately (35x43, “200”) 7. Centre the tube to the erect bucky 8. Set the correct SID (150 cm) Total marks obtained:/8 General comment on preparation abilities of the student: Actual mark Space for narrative as well Space for comment on each point Segment of the Mark sheet

17. Mark sheet continued: If a repeat examination would have been necessary, indicate the reason from the possibilities listed below and subtract 10% from the final mark. Fault Please tick No anatomical marker Incorrect marker Incorrect placement of marker Incorrect SID Tube not centred Entire anatomical area not included Any other:

18. Mark sheet cont. Any areas for improvement? Please comment: Any aspects that were done well? Please comment: Examiner’s signature: _______________________________________

19. Reflections: This assessment is for summative purposes- students like to see their marks This assessment is for summative purposes- students like to see their marks The examiner is not bound by the tick box and is free to write observations which is useful for the student and examiner when giving feedback The examiner is not bound by the tick box and is free to write observations which is useful for the student and examiner when giving feedback Students can see where they went wrong Students can see where they went wrong Students can see where they need to improve Students can see where they need to improve Students can see where they performed particularly well Students can see where they performed particularly well

20. Drafting criteria for rating a clinical skill (STSAF) 1.Decide on the type of student to be assessed 2. Decide on the setting for the assessment 3. Select the clinical skill you want to evaluate (Adapted from Watts, N: 1990) 4. Write down the performance criteria you want to see a student do to demonstrate acceptable skill 5. Discuss, revise and select a master list of criteria

21. Essential elements in the construction of the instrument Inter-rater reliability Inter-rater reliability Construct validity Construct validity 10/10 1 st year: 42% 2 nd year: 63% 3 rd year: 88%

22. Essential elements in the construction of the instrument Aspects of “performance” Observable -(things you can see, hear or feel someone do); Observable -(things you can see, hear or feel someone do); Essential – not simply desirable, or characteristics of an excellent performance, but necessary in order to be acceptably safe, effective and/or Essential – not simply desirable, or characteristics of an excellent performance, but necessary in order to be acceptably safe, effective and/or efficient; Directly related to this skill; Directly related to this skill; Likely to be called for under most (routine) circumstances when this skill is employed (Watts, N: 1990)

23. Other considerations: How often we need to see a student attempt the task before we are prepared to judge their level of mastery; How often we need to see a student attempt the task before we are prepared to judge their level of mastery; What types or degrees of error would say mastery is not yet acceptable; What types or degrees of error would say mastery is not yet acceptable; The importance of procedural steps v speed and smoothness; The importance of procedural steps v speed and smoothness; Routine v unfamiliar situations; and Routine v unfamiliar situations; and How many times until we are convinced of mastery? How many times until we are convinced of mastery?

24. Discussion points What type of instruments are you using to assess the practical competence of your students? What type of instruments are you using to assess the practical competence of your students? What aspects (technical/procedural/ professional) are you assessing? What aspects (technical/procedural/ professional) are you assessing?