Formulating The Method Chapter 4 Formulating The Method Research Methods in Physical Activity

Methodology Methodology of Research must address: 1. Participants 2. Instruments or apparatuses 3. Procedures 4. Design and analysis The purpose of the method section is to explain how the study was conducted. The standard rule is that the description should be thorough enough for a competent researcher to reproduce the study. Research Methods in Physical Activity

Planning is Important The purpose of planning the method is to eliminate any alternative or rival hypotheses. This statement really means that when you design the study correctly and the results are as predicted, the only explanation is what you did in the research . In research we want to use the MAXICON principle: Maximize true variance, or increase the odds that the real relationship or explanation will be discovered; minimize error variance, or reduce all the mistakes that could creep into the study to disguise the true relationship; and control extraneous variance, or make sure that rival hypotheses are not the real explanations of the relationship. You will discover that in the statistical models there are two types of variance: explained and unexplained. The explained variance is the observed changes in the dependent variable scores that occur because of the introduction of the independent variable. The unexplained (error) variance is the observed changes in the dependent variable scores that cannot be explained by the independent variable. Research Methods in Physical Activity

General Principles Two principles that make good sense when planning experiments. 1) Less is More: Carefully evaluate the number of independent and dependent variables that are practically and theoretically important to your study. Use the minimal number of variables you will need to address the problem of the study. 2) Simple is Better: Keep your study straightforward so that when you find something, you can understand and interpret what it means. Understanding your data is an important concept. Although fancy statistical procedures are impressive and informative, there is no substitute for plotting data graphically and evaluating it carefully. Research Methods in Physical Activity

Describing Participants This section of the method of a thesis or dissertation describes how and why the participants were selected and which of their characteristics are pertinent to the study. These are questions to consider when selecting participants: ♦ Are participants with special characteristics necessary for your research? – Age (children, elderly) – Sex (females, males, or both) – Level of training (trained or untrained) – Level of performance (experts or novices) – Size (weight, fatness) – Special types (athletes, cyclists, runners) ♦ Can you obtain the necessary permission and cooperation from the participants? ♦ Can you find enough participants? Research Methods in Physical Activity

Describing Participants In experimental research the interaction among participants, measures, and the nature of the treatment program is essential in allowing the treatment program to have a chance to work. Example: If you select participants who have high levels of physical fitness, subjecting them to a moderate training program will not produce changes in fitness. Thus, you will need to have a working knowledge of the types of individuals and protocols that should respond to the introduction of the independent variable(s). This is possible if you have studied the research related to your proposed study. Research Methods in Physical Activity

Writing About Participants The exact number of participants should be given, as should any loss (attrition) of participants during the time of study. The nature of the research dictates the participant characteristics of interest to the researcher. Example: The participant characteristics listed are extremely pertinent in an exercise physiology study but not at all, for example, in a study of equipment used by children on the playground . Protecting Participants (see Chapter 5, not on exam) Most research in the study of physical activity deals with humans, often children, but it also includes animals. Therefore, the researcher must be concerned about any circumstances in the research setting or activity that could harm humans or animals. In chapter 5, “Ethical Issues in Research and Scholarship,” Research Methods in Physical Activity

Describing Instruments Information about the instruments, apparatuses, or tests used to collect data is used to generate the dependent variables in the study. Consider the following points when selecting tests and instruments: ♦ What is the validity and reliability of the measures? ♦ How difficult is it to obtain the measures? ♦ Do you have access to the instruments, tests, or apparatuses needed? ♦ Do you know (or can you learn) how to administer the tests or use the equipment? ♦ Do you know how to evaluate participants’ test performance? ♦ Will the tests, instruments, or apparatuses yield a reasonable range of scores for the participants you have selected? ♦ Will the participants be willing to spend whatever time is required for you to administer the tests or instruments? Research Methods in Physical Activity

Describing Procedures In the procedures section, you should describe how the data were obtained, including all testing procedures for obtaining scores on the variables of interest. How tests were given and who gave them are important features. You should detail the setup of the testing situation and instructions given to the participants (although you may place some of this information in the appendix). If the study is experimental, then you should describe the treatments applied to the different groups of participants. Consider these points when planning procedures: ♦ Collecting the data –When? Where? How much time is required? –Do you have pilot data to demonstrate your skill and knowledge in using the tests and equipment and knowing how participants will respond? –Have you developed a scheme for data acquisition, recording, and scoring? (These are often computer controlled.) Research Methods in Physical Activity

Describing Procedures ♦ Planning the treatments – How long? How intense? How often? – How will participants’ adherence to treatments be determined? – Do you have pilot data to show how participants will respond to the treatments and that you can administer these treatments? – Have you selected appropriate treatments for the type of participants to be used? The procedures section contains most of the details that allow another researcher to replicate the study. Research Methods in Physical Activity

Describing Procedures The procedures section contains most of the details that allow another researcher to replicate the study. The procedures sections should address the following: ♦ the specific order in which steps were undertaken; ♦ the timing of the study (e.g., time taken for various procedures and time between procedures); ♦ instructions given to participants; and ♦ briefings, debriefings, and safeguards. Research Methods in Physical Activity

Avoiding Methodology Error Pilot all procedures – Even if you are using an instrument or method that has been validated, you need to demonstrate that you can use the instrumentation or protocol without error. Every thesis or dissertation proposal should present pilot work that verifies that all instruments and procedures will function as specified on the type of participants for which the research is intended. In addition, you must demonstrate that you can use these procedures and apparatuses accurately and reliably. Pilot work (definition): Verifying that you can correctly administer the tests and treatments for your study using appropriate participants. A note from the author of your text… During our years as professors, editors, and researchers, we have seen abstracts of thousands of master’s theses and doctoral dissertations. More than 75% of these research efforts are not publishable and do not contribute to theory or practice because of major methodological flaws that could have been easily corrected with pilot work. Research Methods in Physical Activity

Design and Analysis Design is the key to controlling the outcomes from experimental and quasi-experimental research. The independent variables are manipulated in an attempt to judge their effects on the dependent variable. A well-designed study is one in which the only explanation for change in the dependent variable is how the participants were treated (independent variable). The researcher explains the proposed application of the statistics. In nearly all cases, descriptive statistics are provided, such as means and standard deviations for each variable. Descriptive statistics are provided for the dependent measures, and the statistics for establishing differences among groups are reported. If correlation techniques (relationships among variables) are used, then the variables to be correlated and the techniques are named. The point is for you to describe your statistical analyses but not to instruct in their theoretical underpinnings and proper use. Research Methods in Physical Activity

Establishing Cause and Effect The establishment of cause and effect in an experimental study is much more than a statistical and design issue. The issue is one of logic. Research Methods in Physical Activity

Establishing Cause and Effect If the null hypothesis is not true, then what hypothesis is true? In science the researcher seeks to explain that certain types of effects normally happen given specific circumstances or causes. For example, an effect may occur in the presence of something but not in its absence. Must causes be observable? If yes, then two criteria are needed to establish cause and effect. First is the method of agreement. If an effect occurs when both A and B are present, and A and B have only C in common, then C is the likely cause (or at least part of it). Second is the method of disagreement. If the effect does not occur in E and F when C is the only common element absent, then C is the cause (or part of it). Research Methods in Physical Activity

Establishing Cause and Effect – Manipulation Effects In any experimental study, researchers must have a plan as part of the methodology to check the manipulation of the independent variable in relation to the dependent variable. Are subjects participating in the independent variable exposure as required in the methodology ? Doing so decreases the error variance. Research Methods in Physical Activity

Establishing Cause and Effect – Avoiding Fatal Flaws The following are some questions that can help to identify fatal flaws: ♦ Are all hypotheses logical when considering theory and study characteristics? ♦ Are all the assumptions that you are making about the study good ones? ♦ Are the “right” participants in sufficient numbers selected for the study, especially in relation to the treatments and measurements used? ♦ Are the treatments intense enough and of sufficient length to produce the desired changes (especially considering the participants selected)? ♦ Are all measurements valid and reliable as well as appropriate for the planned study, especially considering the characteristics of participants and treatment levels in experimental studies? ♦ Are data collection and storage procedures well planned and carefully done? ♦ Are all extraneous variables being controlled? Research Methods in Physical Activity

END OF PRESENTATION Research Methods in Physical Activity