1. Educational Research 102: Selecting the Best Study Design for your Research Question Francis S. Nuthalapaty, MD 2010 APGO Faculty Development Seminar

2. Disclosures No relevant financial disclosures to declare

3. Learning Objectives Describe types of research and study designs Understand the characteristics of a good research question Understand how to convert the research question into a hypothesis Understand how to select the most appropriate study design to test the hypothesis

4. What do we already know? 1. Issenberg SB, McGaghie WC, Petrusa ER, Gordon DL and Scalese RJ (2005) Features and uses of high-fidelity medical simulations that lead to effective learning: a BEME systematic review. Medical Teacher 27, 2, pp 10-28. 2. Fletcher KE, Davis SQ, Underwood W, Mangrulkar RS, McMahon LF Jr, Saint S. Systematic review: effects of resident work hours on patient safety. Ann Intern Med. 2004 Dec 7;141(11):851-7. TRUE FALSE

5. Write an educational research question in which you have interest

6. Types of Research Empirical vs. Non-empirical Basic vs. Applied

7. Empirical vs. Non-Empirical Research Empirical Involves collection of data first hand Non-Empirical No first hand data collection

8. Example: Empirical or Non-empirical? The Effect of House Staff Working Hours on the Quality of Obstetric and Gynecologic Care Bailit, J et al, Obstet Gynecol 2009 OBJECTIVE: To measure the effect of house staff working hours reforms on the quality of obstetric and gynecologic care.

9. Example: Empirical or Non-Empirical? Features and uses of high-fidelity medical simulations that lead to effective learning: a BEME systematic review. Issenberg SB et al, Med Teach 2005 OBJECTIVE: Review and synthesize existing evidence in educational science that addresses the question, 'What are the features and uses of high-fidelity medical simulations that lead to most effective learning?'.

10. Types of Research Empirical vs. Non-empirical Basic vs. Applied

11. Basic vs. Applied Research Basic Results apply to a great many people and situations Results are related to general theory or to a general field of knowledge Results need not have immediate or even clear implications Applied Results are applicable only to a specific group of people in a particular situation. Results are not necessarily related to a broader field of knowledge Results must have immediate and clear implications for practice

12. Example: Basic or Applied? Assessing Vaginal Surgical Skills Using Video Motion Analysis Diwadkar G et al, Obstet Gynecol 2009 OBJECTIVE: To demonstrate the feasibility of using video motion analysis to quantitate a key step of vaginal hysterectomy and define measurable differences between novice and experienced surgical trainees during vaginal hysterectomy.

13. Example: Basic or Applied? Effects of a Depression Education Program on Residents’ Knowledge, Attitudes, and Clinical Skills Learman L et al, Obstet Gynecol 2003 OBJECTIVE: To determine whether an interactive educational program would improve obstetrics and gynecology Residents' knowledge, attitudes, confidence, and skills in caring for depressed patients.

14. Research Methods What is the difference between Qualitative Research and Quantitative Research?

15. Qualitative vs. Quantitative Research Qualitative Descriptive and exploratory focus Used to gain insight into attitudes, behaviors, values Analysis of unstructured information Narrative reporting Quantitative Hypothesis driven Used to identify association and/or causation Analysis of discrete variables Statistical reporting

16. Quantitative Research Experimental & Quasi-experimental Non-Experimental –Causal-comparative –Correlational

17. Quantitative Research Experimental & Quasi-experimental –Assess effect of an independent variable on dependent variables –Comparison of 2 or more groups –Control over ‘treatment’ & measurement –Randomization –Control group

18. Quantitative Research Designs Study Population Treatment Control Randomize Measurement

19. Quantitative Research Designs Study Population Treatment Control Measurement Randomize Measurement

20. Quantitative Research Designs Study Population Treatment Control Measurement Treatment Measurement Control Measurement Randomize

21. Quantitative Research Designs Treatment Measurement Study Population Study Population

22. Quantitative Research Designs Treatment Measurement Study Population Study Population

23. Quantitative Research Designs Treatment 1 Measurement Study Group 1 Study Group 1 Treatment 2 Measurement Study Group 2 Study Group 2

24. Quantitative Research Designs Treatment 1 Measurement Study Group 1 Study Group 1 Treatment 2 Measurement Study Group 2 Study Group 2 Measurement Matching

25. Quantitative Research Non-Experimental –Causal-comparative –Correlational

26. Causal-Comparative Baseline differences are pre-existing –Non-permutable: Ethnicity, Gender –Permutable: Teaching style Determine the cause or consequences of differences Associations can be identified Causality cannot be determined

27. Quantitative Research Designs Exposures Outcome Control Outcome

28. Quantitative Research Designs Exposures Outcome Exposures Control

29. Quantitative Research Designs Exposure Outcome Group 1 Group 2

30. Correlational Research Single group of subjects Describe degree to which 2 or more quantitative variables are related Help explain important human behaviors Predict likely outcomes Identifies associations Causality cannot be determined

31. Correlational Research

32. Correlation Coefficient –Calculated by regression <.35 = only a slight relationship.40 -.60 = possible theoretical value.50 = minimum for crude predictions >.65 = reasonably accurate predictions >.85 = close relationship –Positive correlation = direct association –Negative correlation = inverse association

33. Threats to Internal Validity Subject characteristics Loss of subjects Maturation Repeated measures Statistical regression Investigator bias

34. External Validity Can the findings from the study be generalized to larger populations? –Subject characteristics –Environment (lab vs. natural setting) –Did the subjects act differently b/c they were enrolled in a study (Hawthorne effect)?

35. What are the steps in the research process?

36. The Research Process Define Research Question Define Research Question Conduct Literature Review Conduct Literature Review Refine Research Question Refine Research Question Design Study Design Study Obtain IRB Approval Obtain IRB Approval Collect Analyze Data Collect Analyze Data Disseminate Results Disseminate Results No hole in the literature Hole in the literature

37. FINER Research Questions F – Feasible I – Interesting N – Novel E – Ethical R - Relevant

38. FINER Research Questions Feasible: –Can be investigated with available: Time Money Space Other resources

39. FINER Research Questions Interesting: –Interesting to you –Interesting to others in the field Novel: –Will answers to the question advance knowledge in the field?

40. FINER Research Questions Ethical: –Will any physical or psychological harm come to anyone as a result of the research? Protect participants from harm Ensure confidentiality of data Avoid knowing deception of participants Relevant: –How might answers to this question improve educational practice?

41. What is a research hypothesis and how is it different from a research question?

42. Research Hypothesis A prediction regarding the possible outcomes of the study Wallen & Fraenkel. Educational Research: A Guide to the Process. 2 nd Ed.

43. Research Hypothesis A statement which: –Summarizes the elements of the study The sample The design The predictor and outcome variables Yoder E, MERC Workshop, 2009 AAMC Annual Meeting

44. Research Hypothesis Based upon a FINER research question Simple – one difference or relationship Specific – clearly stated, defined Stated in advance – before data collection Yoder E, MERC Workshop, 2009 AAMC Annual Meeting

45. Research Hypothesis May be stated: –As the alternative hypothesis Non-directional –The difference is unknown Directional hypothesis –Existing studies/data suggest direction of relationships –As the null hypothesis Yoder E, MERC Workshop, 2009 AAMC Annual Meeting

46. Research Hypothesis H a : There is a difference between groups or variables –Female medical students have a different IQ than male medical students –μ f ≠ μ m or μ f > μ m H 0 : There is no difference between groups or variables –Female medical students have a different IQ than male medical students –μ f = μ m Yoder E, MERC Workshop, 2009 AAMC Annual Meeting

47. Example: Hypothesis Improving Resident Competency in the Management of Shoulder Dystocia With Simulation Training Deering S, et al, Obstet Gynecol 2004 METHODS: Residents from 2 training programs were randomized by year-group to a training session on shoulder dystocia (SD) management that used an obstetric birthing simulator or to a control group with no specific training. Both groups were subsequently tested on a standardized SD scenario, and a physician grader rated the resident's performance with a standardized evaluation sheet.

48. Example: Hypothesis What is the research question? What is the null hypothesis? What is the research study design? –Basic or Applied? –Qualitative or Quantitative? –Experimental or Non-Experimental

49. Quantitative Research Designs Study Population Treatment Control Randomize Measurement

50. Small Group Activity Share your research questions Select one research question to convert to a hypothesis Select the most appropriate study design

51. Give Your Research Impact Investigate important questions Connect your study to prior studies and help build the body of work Uses appropriate research design Consult with a biostatistician in the design phase Dissemenate your results

53. Session content, including narrated MS Powerpoint slides available at: http://www.obgynknowledgebank.net