1. Continuing Medical Education Online
Andi Long, Ed.M.
Manager of Distance Learning and CME Online
Harvard Medical School

2. What is CME? Continuing Medical Education
Licensed health care professionals in the US are required to earn CME credit each year in order to keep licensure active
Required number of credits varies by state
HMS DCE is accredited by the Accreditation Council for Continuing Medical Education (ACCME) to provide continuing medical education for physicians.
Each course must go through a review process, meet ACCME’s requirements for CME, and be approved by the accreditation committee for CME.

3. CME Requirements in Massachusetts
ONE HUNDRED (100) CME CREDITS DURING THE TWO YEARS IMMEDIATELY PRECEDING THE BIRTHDAY LICENSE RENEWAL DATE ARE REQUIRED FOR PHYSICIAN LICENSE RENEWAL. THE MAJORITY OF CREDITS MUST BE EARNED IN STUDY DIRECTLY RELATED TO THE PHYSICIAN'S PRIMARY AREA(S) OF PRACTICE.
CREDITS SHALL BE EARNED AS FOLLOWS: 
a minimum of forty (40) credits completed in category 1. (The entire 100-credit requirement may be completed in Category 1).     
and, a maximum of sixty (60) credits completed in category 2. 
ALSO, REQUIRED AND INCLUDED IN THE BIENNIAL 100 CREDITS OF CME: 
two (2) credits of study of the MA Board of Registration in Medicine's regulations (243 CMR ).   Ten (10) credits total of risk management study with a minimum of four (4) credits in category 1. (The remaining six credits may be in category 1 or category 2). 

4. HMS DCE: Historical Perspective
1872: Harvard University approved a plan for a “special course for physicians” to continue their studies while extensively working in clinics and laboratories.
1911: Graduate School of Medicine of Harvard University formally organized
1968: Department of Continuing Education formally named
16 courses offered to 1,203 participants
1981
44 courses offered to 12,000 participants
2008
Over 260 live courses offered
Over 90,000 health care professionals enrolled
Over one million credits awarded

5. Most common: live conference
Ways to earn CME
Most common: live conference

6. Ways to earn CME at HMS in 2008
Live Courses
Over 260 courses
32 new offerings
In-hospital programs
346 programs in 17 affiliated hospitals
Current Clinical Issues in Primary Care (Pri-Med)
7 conferences; approximately 35,000 attendees
Enduring Materials (DVDs), Journals
CME Online

7. A Comment by Professor Summers
“In the current era of technology, we have an unprecedented opportunity, perhaps even a moral obligation, to share information and knowledge with colleagues around the world.”
Lawrence H. Summers

8. Annual attendance at Live CME Courses
Note: 95% of the 60,000+ attendees are from the U.S.

9. Online CME in 2002 6% of physicians were earning CME credit online
Majority of online CME was a videotaped lecture (from a live event) followed by a test
HMS DCE started developing its online program
4 courses launched in 2003

10. Educational Design
Developed for primary care physicians as well as sub-specialists
Covering the most common diseases
Not video-taped: created for an online audience
Case-based and interactive, rich in visual teaching aids
Linked to updated references
Peer-reviewed by experts

11. Updated Educational Design: Presidential Distance Learning Grant (07)
Build community of learners
Asynchronous discussions, comparing answers
Ask the Author
Moderated, asynchronous Q&A sessions
Individualized performance summaries
Incorrect choices summarized and select readings encouraged
Enhanced multimedia
Video introductions, procedures, flash animations
Improved navigation
Learning outcomes evaluation 11

12. Objectives are clearly defined and noted at the start of each case.

13. Presentations are case-based and engaging.

14. Multiple choice questions must be answered correctly
before proceeding.

15. If answered incorrectly,
authors provide explanations for incorrect choices,
and the participant is asked to try again.

16. If answered correctly…

17. …authors provide explanations
for correct choices…

18. …and incorrect choices.

19. Participants can compare their answers.

20. Participants can submit questions by clicking “Ask the Author.”

21. Questions and answers are posted in the course for others to read.

22. Annotated, high-quality images are used to enhance learning.

23. We also encourage authors to have interactive modules.

25. Sample of interactive Flash animation

26. Video sample: doctor with patient

27. At the end of the case presentation, there is a gallery of images…

28. …an individualized study guide…

29. …and references with active links to PubMed are given.

31. Statistics to Date
57 courses available; another 40 courses in active development
20,000 students from 149 countries have enrolled:

32. Is our program effective?
Knowledge Acquisition
Knowledge Retention
Practice
Improvement
Enhanced
Patient
Outcomes 32

33. Pre/Post-Test Comparisons
(Knowledge Acquisition)
Both groups improved by about 14% 33

34. Follow-Up Comparisons
(Knowledge Retention)
The retention rate of the pilot group was higher (94.6% compared to 89.3%).

35. Clinical Practice Surveys
(Practice Improvement)
Added new question to post-course evaluation (beginning January 2010):
I will make changes in my clinical practice as a result of taking this course.
o Strongly Agree o Agree o Disagree o Strongly disagree
Please briefly describe:

36. Do people like CME Online?
Yes!
98% would recommend to a colleague
Number one recommendation for how to improve: add more courses
Hundreds of quotes from satisfied participants
High ratings in evaluation surveys

37. Eduventures Survey: Key Findings
All features achieved a rank average in excess of 4, indicating strong overall satisfaction.
Please evaluate your experience with HMS CME Online
Feature/Characteristic
Rating
The CME website was clear and easy to navigate.
4.46
The course content was clear and well-structured.
4.36
The assessment questions were helpful and appropriate.
4.35
Use of images/graphics was clear and appropriate.
4.27
The course(s) I took was (were) very relevant to my work.
4.22
The amount of text was appropriate.
4.20
The range of CME Online courses was excellent.
4.08
(1 - 5 scale)

38. User Satisfaction Ratings (% of 4 or more on a 1-5 Likert scale)
Question
Original
Pilot
The navigation of the site was clear and easy to follow.
97%*
94%
The content of this course was extremely valuable to me as a medical professional.
87%
90%
The course format was interactive and engaging.
91%
93%
I would recommend this course to a colleague.
This course was as good or better than a traditional, live CME lecture.
57%
77%
*A substantial number in this group had participated in a prior HMS CME Online course 38

39. Comments from Participants
“The course was excellent and worthy of review over and over again.”
“The diversity and cultural history of the patients in every case were informative and realistic.”
“Excellent, up-to-date and succinct material that was a pleasure to interact with online.”
“Harvard's CME is one of the best. I have completed a lot of CME's with other topnotch universities…and Harvard's is at the top.”

40. Business model Tuition-based program
Individual participants pay per course
Tuition price is location-based
Course Directors receive 35% royalty
Goal: move towards group enrollments

41. Staff CME Online: One full-time project manager
One part-time staff assistant
Leadership: Faculty Dean for Continuing Education and Director of Management and Fiscal Affairs
Content
Course directors are recruited from HMS-affiliated hospitals
Reviewers are recommended and earn CME credit
Within department
Marketing
Data entry, finance processing
Tech support
HMS CET Programmer (as needed)

42. Fee Structure Initially $25 per credit hour
Currently: $20 per credit hour
50% discount to emerging nations
50% discount to HMS alumni
Free for participants living in Africa

43. Overall Fee Structure for Online CME, ‘09
Tuition
Number of Sites
% of Sites
Free
180 63
<$ per hour 72 25
$20 per hour 11 4
$21+ per hour 15 6
Other 2
Source: Bernard Sklar, CMElist.com

44. Eduventures Survey
What are your reasons for not participating in HMS CME Online?
Reason
Count
Percent
I thought the cost of the course was too high.
262
56%
I didn’t have enough time.
121
26%
I prefer live CME courses. 70
15%
I was not interested in any of the course topics. 52
11%
I chose to participate in an online CME program offered by a different institution. 41 9%
I experienced technical difficulties with the website.
I prefer other (non-live) CME course formats. 15 3%

45. Pilot Program for African Nations
More than 20,000 participants from 140 countries
However, less than 200 participants from 16 countries in Africa enrolled
In summer of 2008, with permission from the course directors, all of the online modules were offered free of charge to participants living in Africa 45

46. Pilot Program for African Nations
Within 6 weeks, enrollment by colleagues in Africa increased by more than 500%
Before the program: 205 participants from 16 countries in Africa
Currently: 2,262 participants from 37 countries in Africa
No marketing

47. Postgraduate Association
Members receive:
10% discount on a HMS live course annually
4 Free CME Online Courses
Free CD-ROM of a selected Keynote lecture
12-month subscription to a Harvard Health Newsletter
Education from world-renowned teacher-clinicians at HMS
The opportunity to contribute content and be part of an “elite guest faculty”
A Membership Certificate and lapel pin from HMS DCE 47

48. Postgraduate Association
Fee structure:
Annual membership: $199
Lifetime membership: $995
Statistics to date: 1,310 members
1,020 Annual memberships
290 Lifetime memberships
875 US-based members
435 International members 48

49. How has online CME changed?
Today there are about 285 sites offering CME Online
There has been a modest increase in the number of sites offering free CME – from 168 sites (59%)in December 2006 to 180 sites (63%) in July 2009.
Text (with or without graphics) remains the dominant mode, followed by slide-audio or slide-video lectures, case-based interactive and question-and-answer instruction.

50. How has online CME changed?
2008: Live meetings and conferences accounted for 40% of physician-participants
Home study CME (“enduring materials”) and journals accounted for 20.4% of physician-participants
Online CME accounted for almost 41% of physician-participants (compared to 6% in 2002)

51. Physician Usage of Online CME
According to ACCME:
1997: 13,115 physician-participants (0.34%)
1998: 37,879 physician-participants (1.03%)
1999: 79,536 physician-participants (1.79%)
2000: 181,922 physician-participants (3.57%)
2001: 230,055 physician-participants (4.44%)
2002: 329,110 physician-participants (6.08%)
2004: 895,120 physician-participants (14%)
2005: 1,368,335 physician-participants (18%)
2006: 2,184,460 physician-participants (26%)
2007: 2, physician-participants (30.7%)
2008: 4,365,013 physician-participants (40.9%)

52. New Opportunity: Spaced Education
Developed by B. Price Kerfoot, MD at HMS
Participants engage with material in brief intervals over a long period of time
Material is repeated and spaced over time
All via

53. Pilot Study
Is Spaced Ed an effective supplement to a live, lecture-based course?
Eighty-six attendees enrolled in the trial.
Forty-seven of the participants (66%) completed the program, of whom 42 (89%) submitted the end-of-program evaluation.

54. Pilot Study: Spaced Ed Program
The 12-item adaptive spaced education course was structured as follows:
► Learners were sent two questions every two days.
► If a question was answered incorrectly, it was repeated 12 days later.
► If a question was answered correctly, it was repeated 24 days later.
► Once a question was answered correctly twice in a row, it was retired and not repeated again

55. Spaced Education: The Question55

56. Spaced Education: The Feedback56

57. Pilot Study Results Respondents reported that
The program enhanced the impact of the live CME course (90% strongly agree/agree),
They would recommend the spaced education program to a colleague (85%),
The program reinforced their knowledge of hepatitis C (91%), and
The program increased their confidence in managing patients with hepatitis C (81%).
Forty-one of 42 respondents (98%) requested to participate in further spaced education programs offered as supplements to live CME courses.

58. Comments from Participants
“I found it suited my style of learning well and inspired me to review the course material and read additional literature on Hepatitis C. Wish I didn't need that push but it really helped.”
“I thought it was well-done, very convenient, very well-integrated. “
Spaced Education is “very interesting and easy to fit into [my] schedule.”

59. Spaced Ed Summary
Our study demonstrates that online spaced education is an effective and well-accepted supplement to a live CME course.
Further research is needed to determine if spaced education can boost the ability of live CME courses to improve clinical practice patterns.

60. Summary Physician participation in online CME is increasing each year
Our program evaluations show knowledge acquisition and retention
More work is needed to assess change in practice and improved patient outcomes
We’re continuing to pursue new opportunities: group enrollments, podcasts, Spaced Ed, blended learning, etc.

61. Visit Us and Sample a Case