Continuous knowledge assessment program (CKAP) for surgical residents

Central issues residents get the opportunity to participate in a yearly web based exam on surgery residents get the opportunity to participate in a yearly web based exam on surgery One test: longer training, higher score One test: longer training, higher score Monitor the growth of knowledge of the trainees Monitor the growth of knowledge of the trainees May function as a quality system for surgical care in Europe. May function as a quality system for surgical care in Europe.

What is needed? a large stock of questions will be needed: different areas of surgery different areas of surgery pre- and postoperative care pre- and postoperative care a web based module will have to be made and maintained

Is there an example? American Board of Surgery In-Training Examination (ABSITE) 220 item multiple choice examination developed by experts in surgery representing all of the major American surgical societies (Directors of the American Board of Surgery)

Participation in the ABSITE Pilot in the Netherlands Pilot in the Netherlands 2 academic, 1 teaching hospital 2 academic, 1 teaching hospital Trainee’s 1st – 6th year Trainee’s 1st – 6th year $100 per trainee $100 per trainee One moment for login One moment for login Tough exam (no preparation) Tough exam (no preparation)

Content 250 questions in 5 hours 250 questions in 5 hours Also: Also: GynecologyGynecology CardiothoracicCardiothoracic Tough basic partTough basic part Not included: Not included: fracturesfractures

junior / senior exam

Evaluation by the candidates Number of questions is too much Time for exam is inadequate I had language problems The context of the question was unclear in more than 80% The content covered surgery inadequately More than 20% of the questions was not relevant for surgery The content of the exam was not relevant for my surgical training in more than 20% My knowledge was insufficient to do the exam without further specific studying The exam does not stimulates to enhance my surgical knowledge I would not like to take part in the exam next year voluntarily Next time I won't do better

Evaluation of the candidates Strongly disagreeDisagreeYes/noAgreeFully agree Number of questions is too much Time for exam is inadequate I had language problems The context of the question was unclear in more than 80% The content covered surgery inadequately 6187 More than 20% of the questions was not relevant for surgery The content of the exam was not relevant for my surgical training in more than 20% 7357 My knowledge was insufficient to do the exam without further specific studying 1399 The exam does not stimulates to enhance my surgical knowledge I would not like to take part in the exam next year voluntarily Next time I won't do better Time adequate:90% Language ok:70% Context clear:90% Relevant:55% Study neccess.:95% Voluntarily:55%

Evaluation of the candidates

The scores…

juniorsenior USA average 70%69% Dutch average 45% (36-55) 46% (38-54) Score 50%+ Score 45%+ 3/11 6/11 6/15 8/15

synopsis Succesfull pilot ABSITE Succesfull pilot ABSITE Not all items relevant Not all items relevant Selection afterwards?Selection afterwards? Preparation is necessary Preparation is necessary US trainees take 3 monthsUS trainees take 3 months

Next step Other centers participate Other centers participate Including other countriesIncluding other countries Add preparation Add preparation Junior exam in which years? Junior exam in which years? Post exam: excluding less relevant questions Post exam: excluding less relevant questions

???? ABSITE exam will take place in march 2012 Fee to participate is 100 dollar per resident SCORE website access is another 100 dollars per resident Are the fees dependent on the number of participants? Preparation for the ABSITE will take intensive preparation, most VS residents take at least 3 months beside their clinical work. The SCORE website covers at least 70% of the information necessary for the ABSITE: but which subjects are in preparation (I saw for instance that abdominal aneurysms are not covered) and which subjects can only be studied from textbooks. Can subscribers to SCORE get on line support for guidance through the website? The practice questions in SCORE represent the level of knowledge required for ABSITE (?) Which textbooks shall I advice that the residents should buy for preparation of the ABSITE: we would like to participate in only the senior exam? Can we afterwards indicate which questions should be disregarded, because they are not relevant for the dutch curriculum (gynaecology, thoracic surgery)? Can we invite one or more of you for an instruction course for preparing ABSITE? Timing (March 2012) Costs How to prepare? Practice questions Textbooks SCORE website

Hoe verder? Doen we mee in maart 2012? Doen we mee in maart 2012? Hoe gaan de AIOS voorbereiden? Hoe gaan de AIOS voorbereiden? Verplichte deelname? Verplichte deelname?