SIGGINS MILLER REPORT – CURRICULUM EVALUATION DoT Workshop Sydney, May 2014

Methodology ◦ October 2012 – February 2014 ◦ Data sources  Face to face interviews, N=211 respondents, 29 training sites  Online survey, 94 respondents (78% completion)  Document review  Website statistics  Curriculum Analysis by Expert Medical Educator 2

Strengths and Weaknesses  High engagement by TNDs, DoTs, ESOs and Trainees  High congruence between survey responses, consultation findings and document reviews  High confidence that evaluation is representative  Low engagement by clinical supervisors  Estimated 30% participation  Good response rate relative to other colleges (!) 3

 Trainee Objectives  Service Delivery Commitments  High Quality Delivery of Training  Support Under Curriculum  Mentoring  RANZCR Learning Portal 4 FINDINGS and RECOMMENDATIONS

TRAINING OBJECTIVES – Findings  Curriculum has met objectives to a great extent  Articulates knowledge, skills and attributes  Provides significant structure and guidance  Lack of clarity of role of formative assessments (not relating to summative assessments) 5

TRAINING OBJECTIVES – Recommendations  Increased communication within network participants not involved in NGC meetings  Circulation of minutes / summaries  Open and inclusive meetings  Formal processes for network participants to provide feedback from clinical supervisors and trainees  Identification and sharing of good practice across networks  Clearer guidelines on rotation  More explicit role in assessment / discussion of ‘trainees in difficulty’ 6

Service Delivery Commitments – Findings  Curriculum requirements can be accomplished with service delivery requirements  Trainees are able to meet service commitments in timely and high quality manner  Formative in-training assessments can be integrated into clinical practice (consider case based discussion) 7

Service Delivery Commitments – Recommendations  Sharing of best practice regarding integration of training program curriculum into everyday clinical practice 8

High Quality Delivery of Training – Findings  Purpose, objectives and requirements of curriculum well understood across network participants  High usage of curriculum to guide teaching  High usage of curriculum guideline by trainees  DoTs provide significantly better quality and quantity of feedback vs clinical supervisors  Too soon to assess impact on exam pass rates  Lack of financial support 9

High Quality Delivery of Training - Recommendations  Ongoing educational/meeting forums  Keep abreast of change  Practical advice/best practice  New DoT/Trainee/Clinical Supervision education and support 10

Support Under the Curriculum – Findings  TNDs and DoTs voice greater support under new training curriculum vs old training system  TNDs and DoTs verbalise usefulness of training in supervision skills, in particular, feedback  Increased workload under new curriculum noted (DoT). Variable clinical supervisor support of DoTs.  80% of network participants feel training curriculum contributes positively to training  Level of structure and guidance  Regular interaction with supervisors  Extensive understanding of curriculum by trainees 11

Support Under the Curriculum – Recommendations  Investigate options to better equip clinical supervisors who do not feel supported by the curriculum  Quick reference guide to curriculum  Clinical Supervisor targeted roadshows  DoT Train the Trainer  Promote importance of teaching and supervision in the curriculum (influence future generations of radiation oncologists)  DoT led national or network level meetings with clinical supervisors to discuss integration of curriculum  Investigate trainee views of formative assessments  Continuing refinement of ESO role (vs DoT role) 12

RANZCR Learning Portal – Findings  Enthusiastic anticipation of TIMS  Limited enthusiasm re LMS  Difficult navigation  Non intuitive 13

Work Readiness – Findings  Curriculum based on best practices in education and training, therefore ideally suited to produce work ready trainees 14

Work Readiness – Recommendations  Strengthening non medical expert roles of  Health Advocate  Manager  Scholar 15

So …. Where to From Here?  Preliminary discussion at ETC  RANZCR/Chief Censor/Sandra Turner/Bianca Heggelund planning meeting  Workshop planning completed  Workshop scheduled for 27 June  Attendees:  RANZCR – Bianca Heggelund / Scott Raymond (Head of Speciality Training)  ETC Members  Sandra Turner (Immediate Past Chief Censor) 16

So …. Where to From Here?  ETC workshop agenda  Training Networks and Curriculum Evaluation  ETC workshop outcomes  Formation of project based working groups  Representation from ETC  Representation from ETC Committee Reports  ROTC, TNDF, TRE, AE and RORC  Other experts (We Want Your Help) 17  Review of Key Findings under each outcome  Review of Summary of Recommendations for each outcome (approx. 30 mins in small groups)  Report back to whole group for input/prioritisation  Process and time-line for addressing  Identify people/groups to action

18