THE SIMBASE PROJECT A SUMMARY James Ansell

OVERVIEW OF PROJECT 6 STAGES  WP1MANAGEMENT  WP2DEVELOPMENT OF IAM  WP3PILOT OF ICT BASED SIM ADOPTION  WP4IMPACT MAXIMISATION  WP5GENERAL DISSEMINATION  WP6QUALITY AND EVALUATION MX

WP2 IMPACT ASSESSMENT MODEL AUTHORSUlf-Daniel Ehlers Tatiana Shamarina Heidenreich 41 pages long!!! The model was designed according to 4 cornerstones 1. Identifying criteria to describe impact of simulation training 2. Identifying success factors and indicators of this 3. Defining learning phases to follow during modelling activities 4. The selection of a set of standards MUST HAVE A STEP BY STEP DESCRIPTION OF HOW TO USE IT

WP2 IMPACT ASSESSMENT MODEL 1. Identifying criteria to describe impact of simulation training* 2. Identifying success factors and indicators of this 3. Defining learning phases to follow during modelling activities 4. The selection of a set of standards *From literature, partners practice, requirements of model (from they conclude model IAM has to represent teaching and learning process as a flow of phases) From, this the PRIME model of learning has been adapted

WP2 IMPACT ASSESSMENT MODEL  They conclude that: “The adoption of the IAM can improve current organisational practices and support innovation in the health care sector” “Also, model can be used to understand how medical students and employees develop and exploit their competences”

WP2 IMPACT ASSESSMENT MODEL  This 1st IAM will be refined based on validation results  Objectives of this document:  Define SBT and methods and chose one model for SIMBASE  Analyse parameters which influence success / failure  Define methodology to analyse the way SIM contributes to the learning process  Define quality of learning process

WP2 IMPACT ASSESSMENT MODEL  Current methods available to analyse SIM training  Kilpatrick model  Miller model  PRIME model  Process model ISO/IEC model  Portfolio methods  Assessment methods

WP2 IMPACT ASSESSMENT MODEL Kilpatrick model  Assesses response and impact of an educational exercise Miller Model Pyramid of competence, useful foot mapping assessment methods Prime Model  The one that they have followed closely

 PRIME MODEL  Level 1-7  1 = Participants  2 = Assessment of educational activities  3 = Knowledge acquisition and attitudinal change  4 = Competence  5 = Performance  6 = Patient Health  7 = Community Health WP2 IMPACT ASSESSMENT MODEL

 Process Model  To establish details of quality development  Model can be used for design and structure of training  Can be divided into:  Creating context specific quality profile  Specifying the individual process descriptions

WP2 IMPACT ASSESSMENT MODEL  They use this evidence to present 2 models for simulation  Model 1 – Process model, to understand how to plan  Model 2 – PRIME model, how to characterize competency and show different levels of learning different project participants can how Adapted to the aims of the SIMBASE project

WP2 IMPACT ASSESSMENT MODEL  PROCESS adapted model Planning phase Design Phase Implementation phase Evaluation

WP2 IMPACT ASSESSMENT MODEL  PRIME adapted model

WP2 IMPACT ASSESSMENT MODEL

WP3 PILOTING GUIDE  Outlines how the IAM can be implemented (43 pages!!)  Aims of WP3  Test IAM as a strategy design and assessment instrument  Define patient simulation training strategies  Implement priority actions from these  Access impact of actions on health care systems

WP3 PILOTING GUIDE  Methodology and timetables for pilots  Tools for recording the results of pilots  Types and methods for analysis  Guidelines for piloting co-ordinators in each location  Detailed success criteria for determining models suitability

WP3 PILOTING GUIDE  BEFORE (Planning & design phase of process model)  DURING (3 RD Phase of process model)  AND AFTER TRAINING

WP3 PILOTING GUIDE Before training:  Detect training requirements Appoint panel of experts (5-10) Arrange initial meeting Supply panel with details of discussion before meeting Panel during meeting to be given 5-15 questions Higher the number of participants the fewer the questions

WP3 PILOTING GUIDE Before training: Example questions include: “In your opinion what are the main problems with…?” “If you had to select three aspects that urgently require improvement, what would they be” “What, in your opinion, is the principle requirement of the patients with require to….?”

WP3 PILOTING GUIDE Before training:  Panel should be experts /have extensive knowledge of subject  The panel should contain 2 different profiles:  Scientific technical profile  Administrative profile  Information during meeting should be recorded on audio/video  Results of data gathering should be sent to panel (APPENDIX 1)

WP3 PILOTING GUIDE Before training: Identifying Competencies “Once area that needs improving defined we need to identify competencies and good practises to allow improvement”

WP3 PILOTING GUIDE Before training:  In order to allow this they recommend:  Semi structured interviews with experts (APPENDIX 2)  Duration should be 30mins-1hr  Minimum number of interviews is 5 (reduce false –ve)

WP3 PILOTING GUIDE Before training: Selecting competencies that can be trained & methodology used To selected competencies the following need to be considered: Necessity Impact on health Impact on patient safety Response to simulation-based training Cost-effectiveness of training

WP3 PILOTING GUIDE  To do this need another panel!! (can use previous one)  Identify competencies that can be trained, tools/methods A maximum of one month should elapse between detection of first training requirements and the formation of this panel  Set up of panel numbers etc same as before

WP3 PILOTING GUIDE  Selecting the most suitable methodology and training scenarios  Should consider:  Cost  Applicability  Reduction of learning curve  Allow the evaluation of the acquisition of competencies  High degree of acceptance with trainee  Availability of resources

WP3 PILOTING GUIDE Acceptable types of training to evaluate: Face-to-face training (Role play, debate analysis, brainstorming, Didactic panels) E-training methodology Mixed methodology (face-to-face, e-training methodology) Staged simulation (using actors, clinical interviews) Virtual simulation Robotic simulation

WP3 PILOTING GUIDE  Designing training guide  Once we have detected the training requirements, selected the competencies to be trained, chosen the methodology and scenario the next stage is to: Design the training and produce the training guide

WP3 PILOTING GUIDE Design the training and produce the training guide  Panel should consider the following  Clear selection process of target students  Competencies to be trained  Requirements that will be covered  Final objectives  Methodologies used  Schedule  Evaluation system  Description of feedback procedure to training team  EXAMPLES IN APPENDIX 3 AND 4

WP3 PILOTING GUIDE  Selecting the training team CV screen (teaching ability, training skills etc) Personal interview Practical testing  Selecting the students need to consider:  Profile  Professional category  Previous experience  Ease of transferral  Heterogeneity  Need to evaluate prior knowledge & motivation of students (APPEN 9 & 10)

WP3 PILOTING GUIDE DURING THE TRAINING Really give information about how to run an event! Which WIMAT do all the time No strict protocol to follow in this section apart from: The trainers must: Encourage processes of reflection Obtains and transmits information Informal communication Transmits support Solves problems Be familiar with a range of teaching methods

WP3 PILOTING GUIDE Should be evaluations should be performed at beginning and end of training of motivation and expectation of to assess if teaching had a positive effect Also satisfaction survey for trainers (APPENDIX 8)

WP3 PILOTING GUIDE AFTER THE TRAINING Facilitate feedback channels using questionnaires (APPENDIX 6, 7 and 8) or using virtual communities Revision of training according to feedback Accredit students with certification

WP3 PILOTING GUIDE AFTER THE TRAINING Pick one of following:  Objective evaluation  Open question test  On-going evaluation (APPEN 11)  In situ test Evaluate transfer of skills (recommend during this by self questionnaires to participants) Questionnaires should be completed within 2/52 to 2/12 IMPACT: To establish if students motivate colleagues to come to course?

SUMMARY OF SKYPE MEETING  Stakeholders information to be completed  Before Portugal meeting  Familiarise with WP2 and WP3  Meeting to discuss courses to pilot

QUESTIONS