REFORMATION in SURGICAL EDUCATION & TRAINING Stephen Tobin DEAN of EDUCATION Royal Australasian College of Surgeons IMELF Calgary 2013

REFORMATION ●‘… the action or process of reforming an institution or practice …’ ●From ‘reformare’ = shape again ●The Reformation (the Church 1517) ●Re-formation = the action or process of forming again * NEW OXFORD DICTIONARY OF ENGLISH

SURGICAL EDUCATION & TRAINING ●RACS commenced ●‘Excellence in surgical education & practice’ ●Training in hospitals (employers) ●Framework of SET principles ●Collaboration with Speciality Societies ●Fellowship exam (SET5-6) ●Professional development role as well

The structure of the RACS program ●The College is accredited as the training provider ●The College provides curriculum resources, particularly for the non-technical competencies ●Each of the nine surgical specialties develop and deliver much of their curriculum in the regions and hospitals, and at specialty meetings

SET 2007….. ●Structured program from PGY 3-4 ●Effectively PGY3-6+ ●5-year programs (5-7) ●Absence of official surgical framework for clinical years before SET program ●Workforce issues ●IMGs

The structure of the RACS program ●RACS as an organisation – diversity & number Figures at the end of December 2012 Australia and New Zealand SpecialtyTraineesFellows Cardiothoracic Surgery34182 General Surgery Neurosurgery53229 Otolaryngology Head and Neck Surgery81497 Orthopaedic Surgery Plastic and Reconstructive Surgery83440 Paediatric Surgery22104 Urology Vascular Surgery42191 Totals

The structure of the RACS program ●RACS as an organisation – size and distribution

Issues with SET – reform? ●Medical students: surgical societies ●Numbers of graduates ●Lack of structure PGYs ●Selection ●Supervision – feedback, assessment ●Fellowship exam success % less ●Competent not confident Fellows

MEDICAL STUDENTS ●Well-developed ‘surgical societies’ ■ All 22 medical schools ●Articulate ●Connections – locally, College ●Up to 25% interest ●Courses, evenings, draft agreements….

GRADUATE NUMBERS ●22 MEDICAL SCHOOLS ●55% graduate-entry ●Nomenclature MD increasing ●Numbers 1800 (2005) >> 3800 (2017)* ●Vocational stream jobs less (est 2800 now)* ●Urban v. regional & rural ●IMGs *MTRP 16 TH REPORT, JULY 2013 Australian figures; NZ better match

J-Doc : New for early years ●Suite of resources / e-portfolio ●Assist with pathway to surgery (SET) ●Mapped to AustralianCurriculumFramework* ■ For young doctors in PGY1-2 ●Develop to novice-intermediate levels of the 9 RACS Competencies ●Provide well-developed doctors for SET ■ Or other (procedural) specialties *3 RD EDITION, 2012,

Bringing this together : First profile 12

Education across the continuum ●Developing expectations of behaviour and performance

Education across the continuum

Bringing it all together Social tools/APPS Social tools/APPS Web Linked Resources The PGY Curriculum Passing the GSSE Examination Register for the GSSE examination Links to College documents (policies/competencies) Self Assessment Surgical SkillsBox Practice Bank MCQs Clinical Decision Making Goal Setting e-Learning PRE-VOCATIONAL LANDING PAGE PRE-VOCATIONAL LANDING PAGE Presentations External Web Links Register for Skills Training Courses Apply to SET How to pass the GSSE examinaton Video tips

Program resources…

Digital College Vision

SET selection ●Curriculum vitae (15-30% weighting) ■ Points ■ Prerequisite jobs* ●References: 2 or more…6-8 (27-60%) ■ Some closer to work-based assessment ■ Features of MSF-360 ●Interview (25-45%) ■ Use of the above to rank for interview *some year of application (to come) *others must have been completed

Summary of significant selection-assessment relationships ExaminationsDOPSMiniCEXEnd of Term Assessments SSE Gen- eric SSE Spec- ialty Spec- ific Clin- ical DOPS 1 DOPS 2 DOPS 3 DOPS 4 DOPS Mean Mini- CEX 1 Mini- CEX 2 Mini- CEX 3 Mini- CEX 4 Mini- CEX Mean ETA 1 ETA 2 ETA 3 ETA 4 ETA Mean n CVNeg RR Inter- view Total Sel- ection P < 0.05P < 0.01P < 0.05

What about selection? ●British study compared 1990 cf late 90s ● Compared regional vs.hospital/surgeon selection ●No difference between ‘performance’ ● Dropping-out; length of time; GMC ●Review BJS 2012* ■ No discriminators ■ Psychometric testing some role ● Not statistically significant *Note Leading Articles series 2013 (100 th )

SET PROGRAM - assessment ●Evaluation forms ●LogBooks ●MiniCEX used by 6 (usually early years) ●DOPS used by 7 (as above) ●MSF – 360 noted, not used ●Purpose of assessment ●Signing off for FEX exam.  Collins JP, ANZ J Surg June 2013

SUPERVISION ●“Good supervision is difficult” ●Need to improve quality, numbers ●Academy of Surgical Educators ■ ‘professional development of educators’ ●Foundation course for educators/supervisors ●Authentic ‘signing-off’ ●EPAs

What about the program? ●Academic enhancement (<10%) ● RESEARCH – before or during SET ● Masters programs – Australia-NZ, Uni Edinburgh ●Personal leave (2-5%) ●Probation (2-5%) ●Dismissal (1%) RACS data Comparable ●Time-expired may increase ● As Fellowship pass rate has declined

Fellowship (FEX) examination ●Signing-off is vitally important ●FEX tests : ■ Medical and technical expertise ■ Decision-making and judgement ●Recent close-marking ■ Documentation ●Recent decline in success rates ■ 81% : 63% - May 2013 ■ Variation across states & NZ ■ Variation between specialties

Evaluation of SET – current survey ●Entry level varies ●Need to develop WBA approach ●Working hours ●Presence of ‘Fellows’ ●Signing off? ●FEX approach ■ Surveys of trainees, supervisors, new Fellows

New Fellows ●Competent level across competencies ●What done this mean? ●Need for support at transition ●Matching to positions ●Overseas work ●Post FRACS ‘Fellowships’ ●Work in hospital unit or (private) practice

Reformation… ●SET is grounded in the community ● Hospitals, Specialty societies, College, Urban& Rural ●Excellence >> befit community needs ● DISTRIBUTION OF FELLOWS ●College covers trainees and Fellows ● EDUCATION & PROFESSIONAL DEVELOPMENT ●NEW J-Doc program starts the approach ● RACS COMPETENCIES

Reformation could mean… ●Better quality applicants ●Better selection ●Better supervision & WBA ●Appropriate “signing-off” ● For some, this could be earlier ●FEX matches competent practice ●Support in workplace to start ●Lifelong learning – e-portfolio/CPD (WBA)

Reformation needs… ●Evidence –base & evaluation ●Good supervision ●Recognition-support of supervisors ●Motivated trainees ●Competency assessment with meaning ■ EPAs, training to match profession (work) ●Leadership & awareness of surgical education ■ International influences ■ THANK-YOU