Wednesday, November 5 Evaluating research methods to (1) determine the strongest sides of scientific arguments and (2) generate content for the body of position papers IPHY 3700 Writing Process Map

Process Activity: Evaluating research methods to choose sides and generate content 1. Stay focused on your rhetorical goals to guide your critique of research methods. 2. Create a summary table of research methods and results from studies on your issue. 3. Raise diagnostic questions for identifying strengths and weaknesses in research methods. 4. Apply the think-ahead and think-through strategies to answer the diagnostic questions for evaluating research methods. 5. Apply your evaluation of research methods to choose the strongest side of the argument and to generate content that helps you accomplish your rhetorical goals.

Raise diagnostic questions for identifying strengths and weaknesses in research methods Some (But Not All!) Key Diagnostic Questions 1. How appropriate were the subjects' characteristics? Did the researchers screen the subjects appropriately? 2. Were subjects assigned to groups and conditions without bias? 3. Did the study include a sufficient number of subjects? 4. How appropriate was the study design? 5. How valid and comprehensive were the independent variables? 6. How valid, reliable, and comprehensive were the dependent variables? 7. How effectively did the researchers control for extraneous, or confounding, variables? 8. How fitting and accurate were the study's statistical tests? Handout: Evaluating Research Methods

Apply the think-ahead and think-through strategies to answer the diagnostic questions for evaluating research methods Diagnostic Question #4: How appropriate was the study design? Cross-sectional designs generally have these advantages: 1. They are easy, quick, and inexpensive. 2. They usually don't pose any ethical and physical danger to subjects. Lane et al. Marti et al. Cross-sectional designs generally have these disadvantages: 1. For various reasons, including the ones listed below, they can't determine cause-effect relationships. 2. They may be flawed by selection bias. 3. They may be flawed by response bias and participation bias. 4. They may be flawed by recall bias (when data are derived through self-report surveys). Hints for learning about methodological strengths and weaknesses associated with the study design: 1. Study your research methods! 2. Use the articles on your research issue to help you identify strengths and weaknesses in study design (as well as other aspects of research methods).

My Draft (265 words, still needs work!) Lane et al.'s results and conclusions are limited by problematic methods involving subject selection and the study's cross-sectional design. At one point in time, Lane et al.'s subjects reported their running history (total number of miles and number of years that they ran), and they underwent x-ray exams of the knees and spine. Between the runners and nonrunners, no significant differences existed in the severity of markers of osteoarthritis, including bone spurs, sclerosis, and joint space width. These results would strongly support the claim that running does not increase the risk of developing osteoarthritis, if the runners truly represented the entire population of individuals who have run very long distances over many years. Consider, however, the possibility that Lane et al.'s runners, whose mean age was 57.2 years, represented a unique subset of the population who possess physiological characteristics that enable them to run long distances without experiencing cartilage deterioration. For example, the runners in Lane et al.'s study might have been genetically predisposed to having healthy cartilage (LSG: need to find out about genes and cartilage to develop this section). In this feasible scenario, Lane et al.'s cross-sectional design might have problematically eliminated a large subset of the population of runners who cover extreme distances at a relatively young age, develop osteoarthritis, and then stop running to avoid pain and further cartilage deterioration. If this subset of the population exists and had been included in Lane et al.'s study, the results might have indicated a greater incidence of severity of osteoarthritis markers in the runners versus the nonrunners.

Raise diagnostic questions for identifying strengths and weaknesses in research methods Some (But Not All!) Key Diagnostic Questions 1. How appropriate were the subjects' characteristics? Did the researchers screen the subjects appropriately? 2. Were subjects assigned to groups and conditions without bias? 3. Did the study include a sufficient number of subjects? 4. How appropriate was the study design? 5. How valid and comprehensive were the independent variables? 6. How valid, reliable, and comprehensive were the dependent variables? 7. How effectively did the researchers control for extraneous, or confounding, variables? 8. How fitting and accurate were the study's statistical tests? Handout: Evaluating Research Methods

Independent, Dependent, and Extraneous (Confounding) Variables