Medical Education: An overview of the journal and field of research Kevin W. Eva Editor-in-chief Program for Educational Research and Development Department of Clinical Epidemiology and Biostatistics McMaster University

What is medical education research? Difficult to defineDifficult to define

Influence of students on patient care Influence of students on patient care Assessment of demands students place on faculty Assessment of demands students place on faculty War as an educational experience War as an educational experience Influence of legislative changes in drug promotion on medical students Influence of legislative changes in drug promotion on medical students Selection and recruitment Selection and recruitment Peer assessment Peer assessment Delivering feedback Delivering feedback Validity testing of assessment instruments Validity testing of assessment instruments Student concerns about peer physical examination Student concerns about peer physical examination Can’t be defined by content The most consistent feature is variability Med Educ Vol. 42(12): December 2008

Cohort study Cohort study Correlational study Correlational study Psychometric analysis Psychometric analysis Pre-post quasi- experimental design Pre-post quasi- experimental design Grounded theory Grounded theory Ethnography Ethnography Phenomenology Phenomenology Methodological suggestions Methodological suggestions Critical literature review Critical literature review Can’t be defined by methods The most consistent feature is variability Advances in Health Sci Educ Vol. 12(2): May 2007

My own research interests Selection Instruction AssessmentExpertise Maintenance of Competence Self Regulation

What is medical education research? Difficult to defineDifficult to define ScienceScience "Systematized knowledge derived from observation, study, and experimentation carried out in order to determine the nature or principles of what is being studied.""Systematized knowledge derived from observation, study, and experimentation carried out in order to determine the nature or principles of what is being studied." ( Webster's New World Dictionary of the American Language)( Webster's New World Dictionary of the American Language)

What is medical education research? Difficult to defineDifficult to define Science of medical education researchScience of medical education research "Systematized knowledge derived from observation, study, and experimentation carried out in order to determine the nature or principles of issues relevant to the training and continuing education of health professionals.""Systematized knowledge derived from observation, study, and experimentation carried out in order to determine the nature or principles of issues relevant to the training and continuing education of health professionals."

A tale of two emotions Working in educational research provides:Working in educational research provides: Great joyGreat joy Everyone has an opinionEveryone has an opinion Great frustrationGreat frustration Everyone has an opinionEveryone has an opinion

Goal of medical education research To systematically gather empirical information pertaining to relevant issues to enable more sophisticated opinions to be formedTo systematically gather empirical information pertaining to relevant issues to enable more sophisticated opinions to be formed

“Publish or perish” Survival in academia and advancement in training depend upon publication Survival in academia and advancement in training depend upon publication Possibly more than ever before Possibly more than ever before Size matters! Size matters! Impact factor; citations; h-index Impact factor; citations; h-index Many in our field strive to publish without ever having been trained in research or publication strategies Many in our field strive to publish without ever having been trained in research or publication strategies

What happens to a paper?

Submitted electronically Checked by editorial office Skim read by EiC Sent to Deputy Editor Sent to Reviewers (up to 4) Reviews returned, recommendation madeFinal decision made by EIC Authors informed Rejection Statistical advice

Some consequences Some consequences Outright acceptance without revision extremely unusual Outright acceptance without revision extremely unusual Time-frame given in which to undertake minor revision, reformat, or revise and resubmit Time-frame given in which to undertake minor revision, reformat, or revise and resubmit Revised scripts may be reviewed by EiC, or be sent out to reviewers (same or different) Revised scripts may be reviewed by EiC, or be sent out to reviewers (same or different) Appeals to EiC possible Appeals to EiC possible Will be forwarded to Quality and Standards Group if necessary Will be forwarded to Quality and Standards Group if necessary

Decisions on manuscripts to Medical Education (2007) Total number = 1185 Total number = 1185 Reject without review56% Reject without review56% Reject after review23% Reject after review23% Accept after review21% Accept after review21% Majority of acceptances offered only after revisions are made Majority of acceptances offered only after revisions are made

What makes a good paper? Discuss in small groups

Journal criteria RelevanceRigour OriginalityImportance Written wellEthically sound TimelyInteresting

Critical appraisal: Ask and answer at least the following questions: Who was this written for? Who was this written for? What is the author trying to achieve & why? What is the author trying to achieve & why? What did they do? Why did they do it that way? What did they do? Why did they do it that way? What did they find? What did they find? So what? So what?

The review process

First pass by the EIC Is the paper in journal style? Is the paper in journal style? Read the Guidance for Authors! Read the Guidance for Authors! Is it relevant to the journal’s mission? Is it relevant to the journal’s mission? The ‘So what?’ factor The ‘So what?’ factor Has it been published before (literally or figuratively)? Has it been published before (literally or figuratively)? If your paper has been rejected by another journal, tell the EIC: Be honest & include comments from editor/reviewers If your paper has been rejected by another journal, tell the EIC: Be honest & include comments from editor/reviewers

How does the accompanying letter read? Is it measured and balanced? Is it measured and balanced? Does it ‘sell’ the paper convincingly? Does it ‘sell’ the paper convincingly? Is the paper well-written, proof-read, spell-checked etc? Is the paper well-written, proof-read, spell-checked etc? Is it interesting enough to read to the end? Is it interesting enough to read to the end? Is it the appropriate format/length? Is it the appropriate format/length? Research paper? Short report? Letter? RGS? (TCT?) Research paper? Short report? Letter? RGS? (TCT?)

The key question for reviewers Did you learn something?Did you learn something? Good conceptual frameworkGood conceptual framework Clear methodsClear methods Appropriate analysisAppropriate analysis Accurate discussionAccurate discussion Summarize findings and their relevanceSummarize findings and their relevance Acknowledge limitations of method and extent to which conclusions can be madeAcknowledge limitations of method and extent to which conclusions can be made

Reasons papers get rejected

Most commonly (this year) Not a good fit for the journalNot a good fit for the journal Work doesn’t advance understanding in the fieldWork doesn’t advance understanding in the field Writing trick: Do not mention the local context until the methods sectionWriting trick: Do not mention the local context until the methods section Outcome measures are weak (e.g., satisfaction ratings)Outcome measures are weak (e.g., satisfaction ratings)

The most important part of the paper The most important part of the paper Should be catchy and descriptive Should be catchy and descriptive Different styles for different journals: Different styles for different journals: Declamative Declamative Independent variable, dependent variable, intervention Independent variable, dependent variable, intervention Pose/answer a question Pose/answer a question Titles and Abstracts

Before submission, check the guidelines to authors Before submission, check the guidelines to authors Abstract/summary Abstract/summary Title Title Formatting Formatting Double spacing, tables etc on separate pages, referencing etc Double spacing, tables etc on separate pages, referencing etc Acknowledgements Acknowledgements Conflicts of interest Conflicts of interest Ethics approval Ethics approval

Final thought The Journal’s main responsibilities are towards its readers, not the authors

Medical Education Editor in Chief: Dr Kevin Eva Monthly journal 2007 published 213 papers (not including editorials and letters) The Clinical Teacher Editor in Chief: Professor John Spencer Quarterly journal 2007 published 64 papers ( not including editorials and letters) Contact: