Phase 1 Examination

Purpose of the exam Drive factual learning Confirm trainees’ ability to apply learning Assess ability to think on the spot – Solve problems encountered in the workplace Demonstrate competence in basic oncology sciences Standardised, centrally-marked assessment – “Quality Control”

Exam personnel 10 Assessors (Radiation Oncologists) 10 Assessors (Radiation Oncologists) Subject specific expert panels PhysicsAnatomy Radiation &Pathology Cancer Biology Subject specific expert panels PhysicsAnatomy Radiation &Pathology Cancer Biology Review panel Assessment Executive and invited members Review panel Assessment Executive and invited members Chief Censor Assessments executive Phase I leaders Oversight and co-ordination

Assessor Role Assessors, who are Radiation Oncologists, are no longer required to be subject specific to reflect the clinically integrated nature of the exam. Radiation Oncologists who are appointed as assessors are responsible for the – Writing of the questions and criteria for the phase I exam. – Revision of the questions and criteria based on feedback from expert panels, review panel and Ph1 leaders. – Marking of the exam. – Providing feed back to candidates. – Organisation and administration of the supplementary exam as needed. The 10 assessors will be balanced for experience and geography when feasible. Each assessor to write one question and detailed marking criteria based on – allocated curriculum sections for physics, RCB, anatomy and pathology – Time and marks allocated. Assessors will be provided with the contact details of the expert panel members with whom they can liaise to clarify appropriateness and accuracy of questions and/or marking criteria.

Expert Panel Role Expert panel members are chosen to reflect specific expertise or interest in a subject area and will be led by a Radiation Oncologist. Members of the expert panel may also be appointed as a phase I assessor. The expert panel will: – Assist assessors with question and marking criteria writing. – Form the first phase of the review process to assess the accuracy of the content and to review the appropriateness of time and mark allocation. – Provide feedback to the phase I leaders, reviewers and assessors regarding the phase I exam. – May be called upon to provide feedback for unsuccessful candidates.

Review Panel Role Members of the review panel will be Radiation Oncologists who are members of the assessment executive or invited member/s of the review panel. The review panel is responsible for: – Reviewing the overall appropriateness and fairness of the questions, marking criteria and time allocated. – Providing feedback to phase I coordinators, expert panels and assessors regarding specific questions and overall exam. – Settling disputes regarding the structure, content or execution of the phase I exam which cannot otherwise be resolved

Exam setting process Blueprinting – Aims for broad sampling of the curriculum over time – Considers Previous exams Clinical assignment content Expected level of knowledge (detailed vs general) Writing – Questions and marking criteria developed concurrently – Assessors develop one question each, including ROP, RCB, Anatomy, Pathology parts Review – Exam leaders and Assessors – Expert panels – Assessments Executive Collation and Administration

Exam format 2 papers on 12 September 2014 Each paper – 2½ hours – 5 questions Each includes 4 sub-parts: ROP, RCB, Anat, Path – 30% each for ROP, RCB, Anatomy – 10% for Pathology – Marks clearly allocated in sub-parts – Sub-parts do not rely on each other for successful completion – Each question worth 20 marks 1½ minutes per mark ± Clinical stems No clinical management decisions

Exam format Question style similar to papers Encourage trainees to look for key words – List – Define – Briefly explain – Compare, etc… Include detail appropriate to mark allocation

Exam marking Each question marked by two Assessors – One involved in writing question – One other ‘Checklist’ marking??? – Criteria are set to identify key points – Assessors have discretion to not award marks if candidates do not demonstrate understanding In case of discrepancies – Clerical/procedural check – Third or subsequent marker involvement – Exam leaders, Chief Censor

Passing standard Must pass at least 7 out of 10 questions – Pass mark for each question is 60% (12/20) Must pass each oncology science subject – ROP, RCB, Anatomy each worth 60 marks across papers – Pathology worth 20 marks across both papers – Passing standard for each subject is 50% 30/60 for each of ROP, RCB, Anatomy 10/20 for Pathology Must make a meaningful attempt at all questions

Appeals/borderline cases Appeal process after marks released Candidates may seek special consideration for personal circumstances Supplementary oral exam may be offered if “borderline” (40-49%) in one oncology science subject but all other requirements met

2013 results 31 candidates presented for phase 1 exam 24 met all requirements to pass (77%) – 1 borderline on pathology and anatomy  supplementary viva  passed Final pass rate 25/31 (81%) overall for 2013 – Previous years 2010 – 91% 2011 – 86% 2012 – 81% Outright ‘fails’ demonstrated generalised gaps in knowledge across subjects

Phase 1 exam – messages for trainees Start with the curriculum Use CA’s as a learning guide, but do not limit learning to CA topics Use past papers as a guide Read the question Answer the question asked This is a passable exam!