1. Scholarly Projects: How to be a successful mentor Corey Lyon DO Associate Program Director Kim Insel MD 2 nd year resident University of Colorado Family Medicine Residency

2. Objective Gain an understanding of the role of a faculty scholarly mentor Identify the key EBM components needed to successfully mentor a scholarly project Introduction to a structured model that breaks the writing process into individual steps and overcome barriers to complete the project

3. Requirements Residency Faculty –2 projects per faculty member on average over 5 years Residents –1 project per resident by end of residency “Additionally, all residents must actively participate in scientific inquiry, either through direct participation in research, or undertaking scholarly projects that make use of the scientific methods noted above.” http://www.acgme.org/acWebsite/RRC_120/120_prIndex.asp

4. Faculty requirements Faculty Evaluations –Required annually “These evaluations should include a review of the faculty’s clinical teaching abilities, commitment to the educational program, clinical knowledge, professionalism, and scholarly activities.” http://www.acgme.org/acWebsite/RRC_120/120_prIndex.asp

5. Group assessment Who here is; –Responsible for the resident’s scholarly projects –Has mentored a resident in scholarly writing –Has been a first author on a publication –Has experience with FPIN’s HDA

6. Scholarly activity Boyer’s description of scholarship –Scholarship of discovery New knowledge through traditional research –Scholarship of application Translating knowledge into practical use –Scholarship of integration Taking knowledge from varied sources and connecting them in meaningful ways –Scholarship of education Communication knowledge to learners

7. Mentorship A mentor is; –Someone who takes a special interest in helping another person develop –Possess knowledge and experience –Seeks to help another to optimize an educational experience

8. Mentorship Effective mentoring; –Characterized by mutual respect, trust, integrity –Role model timeliness, organization –Assures the mentee takes an active role.

9. Scholarship mentor Role of scholarly mentor –Experience in scholarship –Teach/reinforce appraisal and writing skills –Integrate process with resident’s overall learning development –Develop life long skills of critical appraisal and how to apply evidence to practice –Takes responsibility for the quality of the manuscript

10. Focus on the goals…. Define your EBM curriculum goals –How to ask an answerable question –Literature searching skills –Grading the evidence –Synthesizing the evidence –Applying the evidence

11. EBM curriculum Need to start with an EBM curriculum –Scholarly projects build on that curriculum Journal club, didactics –Study out of Univ of Toronto 60% of FM graduates report not being well trained in critical appraisal, despite an EBM curriculum Scholarly projects reinforce that curriculum Smith M. Research in residency: do research curricula impact post-residency practice? Fam Med2005;37(5):322-7

12. Utilize FPIN membership Authoring a Help Desk Answer has become part of the curriculum –Achieve goals of scholarly project –Obtain peer reviewed publication Reinforce what was learned Not just busy work

13. 450-600 word manuscript Brief, structured evidence-based answers to clinical questions Work with Local Editor and Editor-in-Chief Peer reviewed at another FPIN program Can be finalized in 6-7 months Published in Evidence-Based Practice and PEPID

15. Approach to mentoring Structure, structure, structure –High yield, high success projects –Break writing into individual skill steps –Relatively quick, attainable projects –Dedicated time Not just on top of all other requirements

16. Structure 4 weeks to complete –Complete during a 4 week rotation –Give ½ day/week of admin time –Outline the timeline 1 st week – Literature review complete –Meet with faculty to review search, decide which article to pull 2 nd week – Review articles –Meet with faculty, discuss article findings, determine which articles to use by relevance, LoE, etc 3 rd week – write outline of manuscript (use table) – meet with faculty 4 th week – Complete 1 st draft – give to faculty with articles

17. Checklist

19. Mentor’s role Set clear expectations for each goal Establish realistic timelines –Negotiate clear deadlines –Use scheduled meetings to ensure steady progress –Contact mentees when they miss a deadline Are they stuck, afraid, overwhelmed, lost, procrastinating. –Be available, timely, model professional communication

20. Mentor’s role Understand balance of educational benefit against residency training –Limited time, experience 1:1 mentoring is instrumental to see to completion –Don’t loose your resident Keep track of goals, know when they have been achieved –When it is time to finish the edits, respond to peer review/editor comments

21. Resident needs Background knowledge –Use project to build on existing knowledge Structure –Timeline, deadlines, time, identifiable goals Time Energy/interest from faculty

22. So, what does it look like… Week 1; defining the question –In order to conduct an effective literature search, you need to know what to search –Breaking the clinical question down assist with the search terms needed –Use PICO model

23. Key ElementsDefine Search Terms Patient / Problem Patient cohort, age, sex Problem, disease, or co-existing conditions. Intervention Proposed drug, therapy, test, intervention etc. Possible prognostic factor, or exposure. Comparison Alternative course of action/inaction? Outcome Goal i.e., relieve or eliminate the symptoms? reduce the number of adverse events? improve function or test scores?

24. PICO Question Example You have a 52 y/o male in your clinic you diagnose with BPH, while discussing treatment options, he asks if Saw Palmetto is a good treatment option a) P: Adult males with BPH b) I: Saw Palmetto c) C: Prescription medications (Flomax, Hytrin, Cardura) d) O: Symptom relief, reduction in symptom score Search terms

25. Week 1 – Smart literature search Smart efficient search –Highly focused (PICO helps) –Most recent evidence (limits on how far back) –Outcome based (POEMS) –Not long and drawn out Let the pyramid be the guide……

26. ..Let the Evidence Pyramid Be the Guide As you move up the pyramid, the amount of available literature decreases, but increases in relevance to the clinical setting. Meta- Analysis Systematic Reviews Randomized Control Trials Cohort Studies Case-Control Studies Case Series Expert Opinion

27. Week 2: Review the articles Decide which articles to use –Assigning a LoE assist with deciding which research articles to use as a reference –Goal of highest level, most recent……. Practice using the CEBM table –www.cebm.orgwww.cebm.org –Based on type of study (therapy, diagnosis) and study design pyramid

28. Oxford Centre for Evidence-based Medicine Levels of Evidence (May 2001) Lev el Therapy/Prevention, Aetiology/Harm Prognosis DiagnosisDifferential diagnosis/symptom prevalence study Economic and decision analyses 1aSR (with homogeneity*) of RCTsSR (with homogeneity*) of inception cohort studies; CDR† validated in different populations SR (with homogeneity*) of Level 1 diagnostic studies; CDR† with 1b studies from different clinical centres SR (with homogeneity*) of prospective cohort studies SR (with homogeneity*) of Level 1 economic studies 1bIndividual RCT (with narrow Confidence Interval‡) Individual inception cohort study with > 80% follow-up; CDR† validated in a single population Validating** cohort study with good††† reference standards; or CDR† tested within one clinical centre Prospective cohort study with good follow-up**** Analysis based on clinically sensible costs or alternatives; systematic review(s) of the evidence; and including multi-way sensitivity analyses 1cAll or none§All or none case-seriesAbsolute SpPins and SnNouts††All or none case-seriesAbsolute better-value or worse- value analyses †††† 2aSR (with homogeneity*) of cohort studies SR (with homogeneity*) of either retrospective cohort studies or untreated control groups in RCTs SR (with homogeneity*) of Level >2 diagnostic studies SR (with homogeneity*) of 2b and better studies SR (with homogeneity*) of Level >2 economic studies 2bIndividual cohort study (including low quality RCT; e.g., <80% follow- up) Retrospective cohort study or follow-up of untreated control patients in an RCT; Derivation of CDR† or validated on split- sample§§§ only Exploratory** cohort study with good†††reference standards; CDR† after derivation, or validated only on split-sample§§§ or databases Retrospective cohort study, or poor follow-up Analysis based on clinically sensible costs or alternatives; limited review(s) of the evidence, or single studies; and including multi-way sensitivity analyses 2c"Outcomes" Research; Ecological studies "Outcomes" ResearchEcological studiesAudit or outcomes research 3aSR (with homogeneity*) of case- control studies SR (with homogeneity*) of 3b and better studies 3bIndividual Case-Control StudyNon-consecutive study; or without consistently applied reference standards Non-consecutive cohort study, or very limited population Analysis based on limited alternatives or costs, poor quality estimates of data, but including sensitivity analyses incorporating clinically sensible variations. 4Case-series (and poor quality cohort and case-control studies§§) Case-series (and poor quality prognostic cohort studies***) Case-control study, poor or non- independent reference standard Case-series or superseded reference standards Analysis with no sensitivity analysis 5Expert opinion without explicit critical appraisal, or based on physiology, bench research or "first principles" Expert opinion without explicit critical appraisal, or based on economic theory or "first principles"

29. CEBM Table: Levels of Evidence (LOE) Therapy/Prevention Etiology/Harm PrognosisDiagnosisDDx/ Symptom prevalence Economic and Decision analysis 1a SR (with homogeneity) of RCT 1b RCT with narrow CI 1c All or none 2a SR (with homogeneity) of cohort studies 2b Individual cohort study or low quality RCT 2c“ Outcomes” research Ecological studies 3a SR (with homogeneity) case control studies 3b Individual case- control study 4 Case series/low quality cohort and case-control 5 Expert opinion w/o critical appraisal, or based on bench research

30. Week # 3 - Synthesis Starts with the statistics Causes the most fear among those new to writing –Critical appraisal skills Emphasize you don’t have to DO the statistics –Just have to understand the statistics

31. Synthesis Utilize an Evidence Based Table to organize the critical appraisal Review table to assure understanding

32. EVIDENCE TABLE What: Tabular description of the studies. Compare studies by characteristics. Why: Prevent errors of interpretation. Increase clarity of analysis. Plan a statistical analysis. Study 1 Study 2Study 3 Citation Population Study Design Intervention (N in the Group) Comparison (N in the Group) Follow-Up Period Outcomes Measure Effect Estimate (CI or p) Study Quality Reviewer Comments Example: Treatment Question

35. Week 4 – rough draft 1 st draft of manuscript to be submitted to faculty mentor by the end of week 4 Faculty reviews, and edits –Responsible for quality If goals achieved, then need to define resident’s role

36. It’s about the goals Project goals –How to ask an answerable question –Literature searching skills –Grading the evidence –Synthesizing the evidence –Applying the evidence Manuscript submitted after all goals achieved

37. Mentor as co-author Important to define roles at this point –Who responds to peer reviewer –Who responds to editor comments –Who follows up to completion

38. Avoid barriers Define when resident is done –Before you lose them Loose deadlines Faculty not engaged –Lose structure Faculty learning while mentoring

39. Summary An active, engaged mentor is key to resident success in scholarly projects Focus on the goals, not the publication –The publication will happen if the goals are achieved Break the writing process into individual EBM steps Utilize a high yield, high success project –FPIN projects (HDAs) –Case studies

40. Questions Other suggestions