Monday, November 12 Evaluating research methods: To determine the strongest sides of scientific arguments and to generate content for position papers IPHY 3700 Writing Process Map

Process Activity: Evaluating research methods to choose sides and generate content 1. Stay focused on your rhetorical goals to guide your critique of research methods. 2. Create a summary table of research methods and results from studies on your issue. 3. Raise diagnostic questions for identifying strengths and weaknesses in research methods. 4. Apply the think-ahead and think-through strategies to answer the diagnostic questions for evaluating research methods. 5. Apply your evaluation of research methods to choose the strongest side of the argument and to generate content that helps you accomplish your rhetorical goals.

Apply the think-ahead and think-through strategies to answer the diagnostic questions for evaluating research methods Diagnostic Question #4: How appropriate was the study design? Cross-sectional designs generally have these advantages: 1. They are easy, quick, and inexpensive. 2. They usually don't pose any ethical and physical danger to subjects. Lane et al. Marti et al. Cross-sectional designs may have these disadvantages: 1. They can't determine cause-effect relationships; they can determine only correlations and general associations between independent and dependent variables. 2. They may be flawed by "self-selection" of subjects. 3. They may be flawed by response bias. 4. They may be flawed by problems associated with self-report surveys. Hints for learning about methodological strengths and weaknesses associated with the study design: 1. Study your research methods! 2. Use the articles on your research issue to help you identify strengths and weaknesses in study design (as well as other aspects of research methods).

My Draft (265 words, still needs work!) Lane et al.'s results and conclusions are limited by problematic methods involving subject selection and the study's cross-sectional design. At one point in time, Lane et al.'s subjects reported their running history (total number of miles and number of years that they ran), and they underwent x-ray exams of the knees and spine. Between the runners and nonrunners, no significant differences existed in the severity of markers of osteoarthritis, including bone spurs, sclerosis, and joint space width. These results would strongly support the claim that running does not increase the risk of developing osteoarthritis, if the runners truly represented the entire population of individuals who have run very long distances over many years. Consider, however, the possibility that Lane et al.'s runners, whose mean age was 57.2 years, represented a unique subset of the population who possess physiological characteristics that enable them to run long distances without experiencing cartilage deterioration. For example, the runners in Lane et al.'s study might have been genetically predisposed to having healthy cartilage (LSG: need to find out about genes and cartilage to develop this section). In this feasible scenario, Lane et al.'s cross-sectional design might have problematically eliminated a large subset of the population of runners who cover extreme distances at a relatively young age, develop osteoarthritis, and then stop running to avoid pain and further cartilage deterioration. If this subset of the population exists and had been included in Lane et al.'s study, the results might have indicated a greater incidence of severity of osteoarthritis markers in the runners versus the nonrunners.

Raise diagnostic questions for identifying strengths and weaknesses in research methods Some (But Not All!) Key Diagnostic Questions 1. How appropriate were the subjects' characteristics? Did the researchers screen the subjects appropriately? 2. Were subjects assigned to groups and conditions without bias? 3. Did the study include a sufficient number of subjects? 4. How appropriate was the study design? 5. How valid and comprehensive were the independent variables? 6. How valid, reliable, and comprehensive were the dependent variables? 7. How effectively did the researchers control for extraneous, or confounding, variables? 8. How fitting and accurate were the study's statistical tests? Handout: Evaluating Research Methods

Independent, Dependent, and Extraneous (Confounding) Variables

Apply the think-ahead and think-through strategies to answer the diagnostic questions for evaluating research methods Diagnostic Question #5: How valid and comprehensive were the independent (or categorical) variables? Do the think-ahead and think-through for a study on the effects of the Atkins diet on weight loss Lane et al. Marti et al.Samaha et al.

Apply the think-ahead and think-through strategies to answer the diagnostic questions for evaluating research methods Diagnostic Question #6: How valid, comprehensive, and reliable were the dependent variables? Lane et al. Marti et al. Do the think-ahead and think-through for a study on the effects of pre-exercise hamstring stretching on injury risk

Apply the think-ahead and think-through strategies to answer the diagnostic questions for evaluating research methods Diagnostic Question #7: How effectively did the researchers control for extraneous, or confounding, variables? Lane et al. Marti et al. Do the think-ahead and think-through for a study on the effects of diet composition versus energy balance on weight loss Golay et al.