SUMMATIVE ASSESSMENT Ramesh Mehay

What is it for? (What does it test?) Summative Assessment OBJECTIVES What is it? What is it for? (What does it test?) Components?

Summative Assessment What Does It Test? objective test of GP registrars' skills. ensures that those completing training achieve a minimum level of competence to practice independently as GP's in the NHS.

Specifc Areas the SA tests? Consultation skills Knowledge Performance as a GPR Ability to evaluate practice / literature / coherent, justified written work Key Areas Being Tested Adequate knowledge Adequate problem solving skills Adequate clinical competence Adequate consulting/comminication skills Demonstration of adequate skills in producing a written report of practical work in general practice Adequate performance on a wide variety of skills, attitudes and knowledge, confirmed by a Trainer's report

Summative Assessment The Components MCQ Written Submission Video / Sim Surgery Trainers Reports VTR forms (1 & 2) THE MCQ There are two types of MCQ that are accepted for the Summative Assessment. You MUST take one of these: the UKCRA MCQ (Summative Assessment MCQ) or the MRCGP MCQ (formerly known as paper 2) The UKCRA MCQ 3 hours long - usually 10am til 1pm 300 items (true false type) and 20 questions of the extended matching type testing knowledge and problem-solving skills contains specific general practice elements (eg admin) Marks are NOT deducted for incorrect answer (ie no negative marking) highly recommended not to take until you have completed 3 months of general practice held on 4 separate occasions each year Dates : early Feb, early May, early Sept, early Dec Venues : Post grad centres of York or Leeds.....on application, the details will be sent to you What If I fail? You can sit the exam at the next available opportunity How Many Attempts Can I Make? Following 2 consecutive failures, issues surrounding failure will need to be discussed between GPR, Trainer, Course Organiser, Summ. Assessment Co-ordinator and Director or Associate Director. However, this does not mean you can't take it again........don't worry, you can, but the discussions will give hopefully put you in a better position of passing. How Much Does It Cost? UKCRA MCQ is free...... How Difficult is the Exam? There is no fixed pass rate. Currently, around 94% of those taking it pass. The pass mark is based on the expected performance of a GPR of minimum acceptable competance. What Sort of Stuff Does it Cover? Internal medicine (medicine, therapeutics, surgical diagnosis, psychiatry, geriatrics etc) - 45% of the total Child Health - 1/6th of the total Women's Health - 1/6th of the total External medicine (ENT, eyes, dermatology) - 1/6th of the total Practice management - 5% of the total If you read the major journals and keep up to date, you shouldn't have any problem. THE VIDEO/SIM SURG The assessment of consultation skills can be taken: · as the Trent Leicester/Yorkshire Simulated Patient Surgery as the consulting skills assessment (video) of the MRCGP as a video-based assessment organised by the UKCRA PLEASE NOTE THAT THE PREFERRED METHOD IN YORKSHIRE IS SIMULATED PATIENT SURGERY (If you wish to submit a video, a blank standard VHS videotape may be supplied by, or requested from, the SA Deanery Office and the beginning of the general practice-based year. This tape can be used for both the UKCRA video and MRCGP 'single route' assessment of consulting skills module. ) If you require more information on the video method of assessment, please refer to the Registrar Handbook. Click here to refer to it online. Simulated Patient Surgery was pioneered by staff in the Yorkshire and South Trent Deaneries and is available to all GP registrars in the UK, with the permission of their Director. THE PROJECT See next slide TRAINERS REPORT What Is the Trainer's Report? The Trainer's report was developed following a national survey of Trainers' views. The standards were produced by a consensus group of experienced Trainers. When these standards are reached the GP Registrar is considered to be ready for independent practice. The report gives guidance to the Trainers on the minimum standards, i.e. what will constitute a pass/failure. The SA Deanery Office issues blank Trainer's reports (including guidance for Trainers completing the report) to GP Registrars and Trainers at the beginning of the general practice based training year. The SA Deanery office will also be able to supply further copies GP Registrars should read the Trainer's report (including guidance for Trainers completing the report) at the beginning of their year and discuss with their Trainer when and how the assessments will be made. When Should the Trainer's Report Be Filled In? It is important that the Trainer's report is completed on a regular, even day-to-day basis with the recording being made contemporaneous with the assessment of the skill or attribute under consideration. It is recommended that some written record be kept of how the assessment was made, and on what grounds the skill and attributes etc were passed/failed. All items in the Trainer's report need to be completed satisfactorily during the penultimate month of the general practice based training year and the report presented 1 month before the completion of the general practice based training year, at the same time as the (final) VTR/1. What Happens If the Trainer Thinks I Have Not Made the Grade? If Trainers have any doubts about whether or not the GP Registrar has reached the necessary standard then the Trainer should seek advice from a Course Organiser, Summative Assessment Co-ordinator, Director and/or Associate Director. Referral of Structured Trainer's Report In consultation with Course Organisers, Summative Assessment Co-ordinator, Director or Associate Director who will recommend what action needs to be taken. What is the Structure of the Trainer's Report? The report is divided into 6 sections: Patient care (divided into general clinical skills, patient management skills and clinical judgement) Communication skills Personal and professional growth Organisational skills Professional values Specific clinical skills (divided into diagnostic skills and emergency care) Trainers are also given guidance as to the best method of assessment under three categories: assessment by observation assessment by discussion assessment by specific methods If a GP Registrar changes Trainer then the final Trainer will seek the advice of the previous Trainer(s) in completing the report. However, it is the final GP Trainer who determines whether the report is to be signed.

Written submission (NPMS) When to do it What to do it on How to write it What standard Choosing method of assessment Who to get help with it from The written submission can be completed at any time during the 3 years of vocational training but submitted no later than 3 months before the end of GP based training year. The ideal time to undertake the project is in the first 6 months General Practice attachment. The Deanery Office can accept submissions at any time but if you are in your final 6 months of training you should submit by: 1st November for 31st January finish 1st May for 31st July finish It is to the GP Registrar's advantage to submit the work as early as possible in case resubmission becomes necessary. The work must be relevant to general practice. GP registrars should remember the importance of presenting original data honestly acquired. Fabricated data will result in local proceedings within the Deanery and could lead to disciplinary procedures via the GMC. Extracts from papers etc should have an acknowledgement of source. Material that is used in submissions and obtained from the World Wide Web without acknowledging the origin of the material and using it for academic purposes amounts to academic fraud. This will also result in local proceedings within the Deanery and could lead to disciplinary procedures via the GMC. .............................be warned! The Written Submission of Practical Work can take one of two forms An Audit - this should be a completed cycle (ie you need to re-audit to see if there was any change. A Project of you own choosing YOU MUST submit 4 copies of the project/audit to the Deanery Office. This should be accompanied by an audit declaration form or the National Project Marking Schedule Application Form which should be signed by your trainer. Please, please please, carefully review the criteria relating to your project - number of words, spacing, anonymity etc. If you do not adhere to the criteria, your project will be returned to you. Why the National Project Marking Schedule (NPMS)? This was originally developed in Yorkshire. It was developed because it was felt that audit alone did not give GPR's the best opportunity to demonstrate their competence in the written submission element of the Summative Assessment. It also provides a greater choice in the type of things you can do and hopefully will mean that you find the written piece both an enjoyable and interesting thing to do. What Types of Things Can I Submit? Any sort of project that interests you. For example : Questionairre study Notes review Literature review Clinical case study Research study Plan for a new service in the practice Audit (must be an 8-point cycle - ask your trainer/Course Organiser what this means) Discussion paper. The marking schedule guides you on what is required to pass. The intention is that this will be an enjoyable and useful experience for you, as well as fulfilling the statutory requirements. If you think you have a good idea for a project but you are not sure that it fits into these categories you should discuss it with your trainer and/or course organiser and then he/she will tell you whether it is suitable. How do you prepare your project? Hints n Tips. Practical Advice: A marking schedule is shown below. As you can see it describes what is needed for each level of achievement. Think of it as the answer sheet to a test! Use it - it tells you what is being examined! When you have completed your project, try and mark it yourself according to the marking grid provided below. This is the grid used by the markers. Please note that a project must score 18 or more (out of 30), with no less than 2 on any criterion, in order to pass. Please look at the marking grid carefully and from this you will see what your project should contain. We would strongly encourage you to write down your aims and what you are going to do as a plan or protocol before you start your project. Examples of projects are available on this website (return to home page and click on "Examples of NPMS Projects"). If you have chosen to submit a project under the National Project Marking Schedule you cannot simultaneously submit under other methods. Please note that the project must be based in, and be relevant to, general practice/primary care and can be completed at any time during your period of general practice training. However, if you want to have the result by the end of your training you must submit it at least three months before the end, to allow enough time should you have to resubmit (i.e. if your project is referred by the second level markers you may be asked to amend and resubmit or undertake a new project). How To Structure Your Project A statement of the aims of the project The background and reason for doing the project should be stated. Reference to the literature relevant to the project The references used should be listed in a consistent style at the end of the project. Normally there would be 6 or more relevant references. They should be referred to clearly in the text. A clear description of the method used to collect evidence The reader should have enough information to be able to repeat what was done. For audit projects, the criteria and standards should be stated. Presentation of the evidence This should be clearly related to the aims of the project, and the method. Consider different ways of presenting numerical data. Discussion of the evidence presented This should consider how well the chosen method addressed the aims of the project, the relevance of the findings to practice, and consideration of change in practice as a result of the findings. Evaluate your work. A conclusion to sum up the points raised in the discussion. Make some practical suggestions to improve a situation. Make constant reference to the marking schedule throughout the planning and writing stages and aim for high marks for each criterion.

Useful Resources for MRCGP Join an MRCGP Group Go on the Yorkshire Preparatory Course MRCGP Newsgroup http://groups.yahoo.com/group/mrcgp-yorks MRCGP website Coming soon at a Computer Near You!

THE SINGLE ROUTE The MRCGP/Summ Assessment Single Route MCQ The MRCGP/Summ Assessment Single Route Video The MRCGP/Summ Assessment Single Route MCQ Candidates who have passed the MCQ component of the MRCGP exam (irrespective of whether they pass or fail the examination overall) will be issued with a letter from the College to this effect. Passing the MRCGP MCQ exempts you from taking the summative assessment MCQ. In such cases, the letter from the college and the VTR 1 from should be sent together to the JCPTGP. The MRCGP/Summ Assessment Single Route Video Candidates who have passed the video component (consulting skills assessment) of the MRCGP (irrespective of whether they pass or fail the examination overall) are exempt from taking the Simulated Surgery component of the Summative Assessment