Impact of e-learning for post-graduated medical education: a systematic review I N T R O D U C T I O N T O M E D I C I N E Authors: Ana Silveira; Catarina Rodrigues; Fábia Carvalho; Joana Vargas; João Faria; Leonor Marques; Luís Braz; Orlando Cardoso; Rui Cerqueira; Sofia Baptista; Sofia Saldanha; Susana Santos. Supervisors: Mário Dinis Ribeiro, MD, PhD; Altamiro da Costa Pereira, MD, PhD /

@ S U M M A R Y I N T R O D U C T I O N T O M E D I C I N E This presentation’s main goal is to give an overview of our study, highlighting it’s final results, conclusions and a brief discussion about the limitations we faced. To achieve such a goal, we opted to keep a global perspective of the study design, by showing the steps we have performed since the background information, research methods, results, conclusions and a final comment on the future perspectives.

McKimm J et al. McKimm J et al. BMJ VOLUME APRIL 2003; Nelson E A, Nurs Adm Q Jan-Mar;27(1): Distance Education is a concept known since the 19th century. With the development of World Wide Web it has taken a large step towards the creation of e-Learning. World Wide Web – Use of the Internet to present various types of I N T R O D U C T I O N I m p o r t a n t C o n c e p t s I N T R O D U C T I O N T O M E D I C I N E

Internet - A global network of computers divided into subsets. e-Learning (web-based learning – WBL) - Online computer (web)- based educational training through electronic means, such as web, intranet, or other multimedia materials. McKimm J et al. McKimm J et al. BMJ VOLUME APRIL 2003; Nelson E A, Nurs Adm Q I N T R O D U C T I O N I m p o r t a n t C o n c e p t s I N T R O D U C T I O N T O M E D I C I N E

Virtual Learning Environments - a set of electronic teaching and learning tools, a platform providing online student support and electronic communication. Medical schools have been implementing Virtual Learning Environments (like “moodle”, for example). McKimm J et al. McKimm J et al. BMJ VOLUME APRIL I N T R O D U C T I O N I m p o r t a n t C o n c e p t s I N T R O D U C T I O N T O M E D I C I N E

Medical education is education related to the practice of being a medical practitioner, either the initial training to become a doctor or further training I N T R O D U C T I O N I m p o r t a n t C o n c e p t s I N T R O D U C T I O N T O M E D I C I N E

@ I N T R O D U C T I O N I m p o r t a n t C o n c e p t s I N T R O D U C T I O N T O M E D I C I N E Vozenilek J, et al. ACAD EMERG MED d November 2004, Vol. 11, No. 11. Medical Education Distance Education (computer-based) e-Learning Examples: Virtual reality (High-fidelity human patient simulator); WBL environments (on-line theorical courses).

To Understand the impact of e-Learning in medical post- graduate education How often is e-Learning used by Medical Doctors? Which are its effects in learning/training? Is e-learning effective? What are participants’ I N T R O D U C T I O N R e s e a r c h Q u e s t i o n s a n d A i m I N T R O D U C T I O N T O M E D I C I N E

Study Design: Systematic Review; Data Base: Medline (PubMed) and Scopus; Target population: Articles about medical post-graduate web based courses; Keywords: “e-learning”; “Web-based learning”; “post-graduation”; “medical education”; “distance M E T H O D S I N T R O D U C T I O N T O M E D I C I N E

Inclusion criteria: - Original Scientific papers/manuscripts assessing e-learning in post-graduate medical education. Exclusion Criteria: - Written in other languages than Portuguese, English or Spanish; - With no information about at least one of these topics: - Frequency of usage of E-learning; - Efficiency; - Effects in clinical performance; - Participants’ M E T H O D S I N T R O D U C T I O N T O M E D I C I N E

Pubmed: (("Education, Medical/education"[MeSH] OR "Education, Medical/methods"[MeSH] OR "Education, Medical/statistics and numerical data"[MeSH]) * ("Education, Distance/classification"[MeSH] OR "Education, Distance/methods"[MeSH] OR "Education, Distance/statistics and numerical data"[MeSH])) OR (("e-learning" OR "web-based learning“ OR "electronic learning") AND “medical education”) 188 articles M E T H O D S I N T R O D U C T I O N T O M E D I C I N E

Scopus: (TITLE-ABS-KEY("E-Learning" OR "web-based learning“ OR "Electronic Learning" OR distance) AND TITLE-ABS-KEY("Medical Education") 45 articles M E T H O D S I N T R O D U C T I O N T O M E D I C I N E

Scopus N = 45 PubMed N = Total M E T H O D S I N T R O D U C T I O N T O M E D I C I N E FluxogramFluxogram (selection of participants) We built a data base on SPSS with the results of the application of the inclusion and exclusion criteria.

Measuring M E T H O D S I N T R O D U C T I O N T O M E D I C I N E Proportion of agreement= (99+106)/225=91,1%

(1) Quantitative results:(2) Qualitative results: a.Year of publication; b.Country; c.Medical speciality. a.Frequency of usage; b.Efficiency of e-learning; c.Effects in clinical performance; d.Participants attitudes: ◦ Positive aspects; ◦ Negative aspects. We added to the SPSS data base the quantitative M E T H O D S I N T R O D U C T I O N T O M E D I C I N E We chose what kind of information would to be extracted from the articles that were analyzed:

a. Year of R E S U L T S Q u a n t i t a t i v e I N T R O D U C T I O N T O M E D I C I N E

b. R E S U L T S Q u a n t i t a t i v e I N T R O D U C T I O N T O M E D I C I N E

c. Medical R E S U L T S Q u a n t i t a t i v e I N T R O D U C T I O N T O M E D I C I N E Hospital Others

a.Frequency of usage: No article was found presenting information about the number of medical doctors using WBL platforms or participating in online courses; 3 articles referred other information under this domain: - Number of users is increasing (2 articles); - Mainly female or young users (1 article); - Number of web-based courses is increasing (1 R E S U L T S Q u a l i t a t i v e I N T R O D U C T I O N T O M E D I C I N E

b. Efficiency of e-learning:  4 articles had information on this domain: - There is an higher knowledge gain in WB learners (3 articles); - Simulators, which create virtual models, were found to be comparable to conventional training (1 R E S U L T S Q u a l i t a t i v e I N T R O D U C T I O N T O M E D I C I N E

 Some study designs: - A pretest/posttest self-controlled study, involving two groups: a WBL and a “face to face” R E S U L T S Q u a l i t a t i v e I N T R O D U C T I O N T O M E D I C I N E Wiecha JM, Chetty VK, Pollard T,Shaw PF. Web-based versus face-to-face learning of diabetes management: the results of a comparative trial of educational methods. Fam Med Oct;38(9):

 Some study designs: - A study used case vignettes to evaluate physicians knowledge of biologic and chemical threats. Responses to clinical vignettes were compared between both groups: participants and R E S U L T S Q u a l i t a t i v e I N T R O D U C T I O N T O M E D I C I N E Casebeer L, Andolsek K, Abdolrasulnia M, Green J, Weissman N, Pryor E, Zheng S, Terndrup T. Evaluation of an online bioterrorism continuing medical education course. J Contin Educ Health Prof Spring;26(2):137-44

c. Effects in clinical performance:  4 articles had information on this domain: - WBL enables doctor to overcome the embarrassment factor (1 article); - students/practioneers that used WBL proved to respond more quickly an appropriately in real clinical situations than those who had learnt from a conventional (“face-to-face”) course (2 articles); - on-line courses increase: ◦ Confidence in treatment; ◦ Awareness in prescribing habits (1 R E S U L T S Q u a l i t a t i v e I N T R O D U C T I O N T O M E D I C I N E

d. Participants attitudes:  11 articles had information on this domain: ◦ 9 articles reefer positive participants’ attitudes: - Students and teachers appreciate these methods’ characteristics (3 articles): ▪ Learning delivery ▪ Ease of access ▪ Availability at any time ▪ Personalized instruction ▪ Interactivity ▪ Good learning interface ▪ Ease in updating R E S U L T S Q u a l i t a t i v e I N T R O D U C T I O N T O M E D I C I N E

@ R E S U L T S Q u a l i t a t i v e I N T R O D U C T I O N T O M E D I C I N E - CRISIS Model: Harden RM. A new vision for distance learning and continuing medical education. J Contin Educ Health Prof Winter;25(1):43-51

@ R E S U L T S Q u a l i t a t i v e I N T R O D U C T I O N T O M E D I C I N E - Besides, e-learning allows to: ▪ Decrease teachers/tutors salaries (4 articles); ▪ Avoid the necessity of teachers/students to travel (4 articles); ▪ Reduce the need for big buildings (4 articles); ▪ Avoid paper based material and books (4 articles); ▪ Decrease in amount of time needed for teaching (1 article)

@ R E S U L T S Q u a l i t a t i v e I N T R O D U C T I O N T O M E D I C I N E ◦ 3 articles reefer negative participants’ attitudes: - Difficulties in implementating e-learning (2 articles); - Low motivation level for those practioneers/students who spend too much time in front of the computer, instead of contacting with patients (1 article).

Family medicine n = 8 Hospital (reproductive medicine; radiology) n = 3 Other (occupational medicine; medical physics) n = 3 Frequency of usage Increase in number of users(1) - There is an increasing number of post-graduate web based courses (1) Efficiency of e-learning Knowledge gains measured in on-line theoretical tests(3) Software based simulators are comparable to conventional training (1) - Effects on clinical performance Enhances practioneers performance measured in practical tests(2) -- Participants Attitudes Positive (4) Negative (1) Positive(2)Positive (1) Negative (1) R E S U L T S Q u a l i t a t i v e I N T R O D U C T I O N T O M E D I C I N E

1.The number of WBL users and courses is increasing, which shows that there is a growing interest on this D I S C U S S I O N I N T R O D U C T I O N T O M E D I C I N E 2.On-line users learn quicker and have a slightly superiority in clinical performance than conventional curricula users; 3.The difficulty about the analysis of the 2 nd conclusion was to explain why this occurs: a.face-to-face learners could reveal a lack of attention during the oral discussion sessions; b.there is a low level of motivation among WB learners that cannot have enough contact with patients.

@ D I S C U S S I O N I N T R O D U C T I O N T O M E D I C I N E 4.Web-based courses are, at least, comparable to conventional ones so instead of deciding which is better, they should be used as complements; 5.E-learning has proved to be a consistent method among the clinical and practical subjects rather than in theoretical ones. Since we know that, in post-graduation, the clinical practice is highlighted, WBL appears more often in post-graduate CME than in basic academic courses;

6.Reliable overall evaluation, based on randomized controlled trials to provide data on the effectiveness of online course intervention is still D I S C U S S I O N I N T R O D U C T I O N T O M E D I C I N E

@ D I S C U S S I O N M a i n L i m i t a t i o n s I N T R O D U C T I O N T O M E D I C I N E The number of studies in this field (e-learning in postgraduate medical education) is reduced; Among these, the number of included articles was low: many of them only describe characteristics and functioning of programs of e- learning rather than their implementation and impact; The language; The articles found revealed to be heterogenic and include much more qualitative considerations than quantitative and statistical data; We didn't have access to 44% of the included articles.

@ F U T U R E P E R S P E C T I V E S I N T R O D U C T I O N T O M E D I C I N E  It would be important to make more studies evaluating the implementation of WBL courses.  We can anticipate that medical education will increasingly take advantage of the benefits of distance learning to improve the quality of post-graduation courses.

A N N E X E S I N T R O D U C T I O N T O M E D I C I N E