Queensland University of Technology CRICOS No J Standardising Radiation Therapy Undergraduate Competency levels Mary-Ann Carmichael Lecturer Radiation Therapy

CRICOS No J a university for the world real R Presentation Overview Rationale Study Aims and Objectives Method Results Discussion Conclusion

CRICOS No J a university for the world real R Background Changes to course at QUT Removal of staged assessment New clinical achievements

CRICOS No J a university for the world real R Rationale Inconsistencies in perceived levels of student competency Student feedback indicates discrepancies in marking and assessing between clinical centres Student feedback also indicates discrepancies in marking and assessing amongst staff in the same clinical centre

CRICOS No J a university for the world real R Aims and Objectives Standardise levels of clinical achievement for each year group Collate staff opinions of levels of student progression Evaluate degree of correlation between staff seniority and perception of student levels

CRICOS No J a university for the world real R Methods Anonymous questionnaire to find out what these perceived levels of student competency were Questionnaire consisted of a series of statements – RTs asked to assign which year level of student should be able to perform each task (in their opinion) The questionnaire was distributed to all RTs in Queensland

CRICOS No J a university for the world real R Results Questionnaires provided to all RTs in Queensland Responses received from 89 RTs of various staff grades: Grade not supplied NPDPHP3HP4HP5HP

CRICOS No J a university for the world real R Start Year 2Year 2Start Year 3Start NPDP Prepare room + patient data for appropriate processes Anticipate sequence of procedures Work as part of the team with minimal direction Can be trusted to work independently Help with basic treatment techniques under close direction Work as part of the team and able to ask for direction when necessary Can be trusted to take on messages and pass on to rest of team Can be trusted to explain procedures to patients and handle all questions asked Follow departmental protocols on radiation safety and H+S Provide sound rationale for their decisions Apply knowledge to non-routine cases Needs constant supervision and direction for all tasks Operate treatment or localisation equipment with minimal direction Create a clinically acceptable plan for a routine site independently Results

CRICOS No J a university for the world real R Start Year 2Year 2Start Year 3Start NPDP Prepare room + patient data for appropriate processes Anticipate sequence of procedures Work as part of the team with minimal direction Can be trusted to work independently Help with basic treatment techniques under close direction Work as part of the team and able to ask for direction when necessary Can be trusted to take on messages and pass on to rest of team Can be trusted to explain procedures to patients and handle all questions asked Follow departmental protocols on radiation safety and H+S Provide sound rationale for their decisions Apply knowledge to non-routine cases Needs constant supervision and direction for all tasks Operate treatment or localisation equipment with minimal direction Create a clinically acceptable plan for a routine site independently Results

CRICOS No J a university for the world real R Results No correlation at all between the staff grade and what they expect students to be capable of Pearson Correlation Coefficient range: –

CRICOS No J a university for the world real R Discussion Limitations High standard deviations of responses –Suggests wide variation of staff perceptions –Further work needed to achieve agreement

CRICOS No J a university for the world real R Discussion “The assessment should reflect the consistency and standard of performance across the placement, taking into consideration the experience level of the student.” Australian Universities Radiation Therapy Student Clinical Assessment Form User Guide – version 1a

CRICOS No J a university for the world real R Discussion “Professional competency is more than factual knowledge and the ability to solve problems with clear cut solutions; it is defined by the ability to manage ambiguous problems, tolerate uncertainty and make decisions with limited information” (Schon, 1983)

CRICOS No J a university for the world real R Conclusions Results suggest considerable variation in staff perceptions of student levels Staff agree that amount of direction is an indicator of level of student ability No correlation between staff grade and perception of student achievement for levels

CRICOS No J a university for the world real R References Australian Universities Radiation Therapy Student Clinical Assessment Form - User Guide – version 1a Baartman, L.K., Bastiaens, T.J., Kirschner,.P.A. and van der Vlueten, C.P.M (2007) Evaluating Assessment Quality in Competence-Based Education – A Qualitative Comparison of Two Frameworks. Educational Research Review 2: Schon, D. A. (1983) The Reflective Practitioner. New York: NY Basic Books

CRICOS No J a university for the world real R Acknowledgements Pete Bridge Queensland RT Clinical Educators Participating Clinical Staff