1. Listening to Our Learners: Designing a Residency Evidence- based Medicine Curriculum Using a Learner-Driven Method Drew Keister, MD 55 MDG Family Medicine Residency Offutt Air Force Base, Nebraska Heath Grames, PhD Assistant Professor University of Southern Mississippi

2. Overview EBM Education Background Summary of the EBM curriculum project Methods for the focus group Focus group results Interpretation Implications for curriculum

3. Background EBM essential to 21 st Century medicine No standardized EBM teaching Prior systematic reviews show: –Poor outcome assessment (Coomarasamy, 2003) –Adult learning theory not addressed (Straus, 2004) –Research driven from academia ? Relevance to practice (Coomarasamy, 2004)

4. The Project Create a learner-driven EBM curriculum Resident feedback informs content: –Needs-assessment survey –Resident-driven curriculum committee –Focus Group Multiple-method study to evaluate

5. The Goal of the Project: THE IDEAL CURRICULUM!!! Address resident-perceived needs Responsive to feedback Input from faculty with private- practice experience

6. Result: Practice Improvement Educational practice improvement Residency graduates trained in evidence-based practice improvement

7. Curriculum Assessment Using Focus Groups Widely accepted tool for curriculum planning and evaluation (Frasier, 1997) Focus Group for EBM Curriculum (Akl, 2006) –Focus-group of 12 volunteer residents from a tertiary care center –Multiple specialties and years of training –Discussion centered on EBM practitioner model vs. EBM user model (aka- Information Mastery)

8. Akl et al Focus Group Participants made the following statements: –“Every doctor should have a high level of [EBM] skill.” –“We can’t always be practitioners... we don’t have all the time to analyze every article.” Majority of participants- practitioner model mandatory Some believed it should be optional

9. Our Focus Group Final stage of needs assessment Research questions: –What priorities do the rank-and-file residents within our program identify as being important for our evidence-based medicine curriculum? –Can we triangulate our data from other needs-assessment methods?

10. Focus Group Methods Qualitative research methods Purposive sample of residents from all class years –Large civilian/military integrated program –Invited all residents enrolled in project Scripted focus group protocol Facilitated by experienced qualitative researcher

11. Focus Group Methods No residency faculty present Focus group digitally recorded Data transcribed & evaluated for clarity –Identifying info removed Template analysis

12. Template for Analysis Definition of Evidence-based medicine (EBM) Important EBM components Barriers to practicing EBM Components important for practice EBM benefits How & When to teach EBM

13. Focus Group Participants 12 invited, 11 attended (92%) 7 male, 4 female Class year- Interns: 6 2 nd years: 4 3 rd years: 1 10 of 11 participants military (91%)

14. Results: Barriers “The statistics killed me on it. It probably killed my drive to find EBM… I don’t have the time to play with that and I don’t particularly enjoy it and it takes the fun out of medicine to me.”

15. Results: Barriers EBM takes away the Art of Medicine: “It pisses me off…because fully 80 percent of the latest research will be either not be reputable in future research or will be disproven.”

16. Results: Key Components “Accessibility. You’ve got to be able to access it and use it.” “Yeah and I like the conciseness... It’s got to be time. It’s got to be worth the time to look at it.”

17. Results: EBM Benefits “We’ve got to teach ourselves how to be self-learners… I mean we can’t just drop it cause if we do, we’ll wither on the vine.”

18. Results: We Don’t Always Like What We Need “You’ve got to have that statistically based baseline. Otherwise, none of it makes sense. You get that in medical school. It sucked.”

19. Results: How to Teach In response to another resident’s concerns about statistics: “And I can’t disagree with that, but I’m just coming at it from a person who is going to be hopefully doing academic medicine and wants to be very good at this and may be writing papers and that sort of thing.”

20. Our Interpretation Residents want to know EBM EBM teaching should: –Be practical –Address how to maintain the art of medicine No technique will work for everyone THE IDEAL CURRICULUM!!!

21. Focus Group Limitations Qualitative research is not generalizable –Unique military-civilian program Demographic imbalance- mostly military This was only one focus group –Acting as triangulation of other data

22. Focus Group Implications Reaffirmed the need for the curriculum –Challenge of learners with different priorities Supported focus on Information Mastery/ EBM User Model Raised my awareness of key concerns

23. Conclusions Resident EBM education is challenging –Residents demand practical skills Focus groups helpful in curriculum design Aim High, but have realistic expectations!

24. Acknowledgement Thanks to: Our co-investigator, Dr. Jenenne Geske at UNMC UNC Fellowship Research Team

25. Questions? Contact Information: Drew Keister, MD drew.keister@offutt.af.mil drew.keister@offutt.af.mil Heath Grames, PhD heath.grames@usm.edu heath.grames@usm.edu