Research Skills Workshop Designing a Project Qualitative: research on unique events; action research. Quantitative: effects; quantitative designs (descriptive, experimental); compliance, bias, placebos; sample size for adequate precision; validity and reliability; females and/or males; individual responses; mechanisms. Will G Hopkins Physiology and Physical Education University of Otago, Dunedin, NZ

There are two approaches: qualitative and quantitative. General Research is the name for the process of finding out about things, especially the relationship between things. There are two approaches: qualitative and quantitative. Use qualitative research to answer "What's happened here?" Use quantitative research to answer "What's happening generally?"

Qualitative Research In its purest form, qualitative research is a detailed examination of a single instance or case of something, usually through intensive interviewing or analysis of other evidence. The quest for truth is similar to that in a court case. Qualitative paradigms that emphasize subjectivity may be closer to art than science. Use it to investigate unique events and to explore potentially important factors for a quantitative study. The open-ended nature of data gathering allows for more flexibility and serendipity in identifying factors and practical strategies than a more structured quantitative approach. Formal procedures (triangulation, member checking, peer debriefing, auditing…) reduce subjectivity.

Action research–a qualitative intervention–can establish causality more convincingly than a purely descriptive qualitative study. You can use qualitative techniques to study more than one case, but any attempt to generalize from this sample to a population is really quantitative research. Samples are usually too small, either because few subjects are available, the data gathering is too time consuming, or researchers don't want to be quantitative. Any study with a small sample can establish only the presence or absence of strong associations in a population. Moderate, weak, or trivial associations remain unclear, because of the uncertainty in the magnitude of these associations. Intensive interviewing is equivalent to assaying many variables, so there is a high risk of finding a spurious association.

Quantitative Research In the simplest quantitative research, the aim is to determine the prevalence, incidence, mean value or other descriptive statistics of something in a population through study of an appropriate sample. More often the aim is to determine an effect: the relationship between the thing of interest (a dependent or outcome variable, such as performance) and other things (predictor variables, such as training, sex, diet). The relationship is expressed as an outcome statistic, such as a difference or change in the mean value, a correlation coefficient, or a relative frequency or risk. You get an estimate the magnitude of the statistic.

Quantitative research designs are either descriptive (subjects usually measured once) or experimental (subjects measured before and after a treatment or intervention). A descriptive study establishes only associations between variables. An experiment establishes causality, or lack of it. Descriptive studies, worst to best: Case, e.g. a gold medallist. Case series, e.g. 20 gold medallists. Cross-sectional (correlational), e.g. sample of 1000 athletes. Case-control (retrospective), e.g. 200 Olympians and 800 non-Olympians. Cohort (prospective or longitudinal), e.g. measure characteristics of 1000 athletes then determine incidence of Olympic medals after 10 years.

Experimental studies, worst to best: No control group (time series), e.g. measure performance before and after a training intervention. Crossover, e.g. give 5 athletes a drug and another 5 athletes a placebo, measure performance; wait a while to wash out the treatments, then cross over the treatments and measure again. Controlled trial, e.g. measure performance of 20 athletes before and after a drug and another 20 before and after a placebo (need 4x as many subjects as in a crossover).

The estimate of the relationship is less likely to be biased (not the same as in the population) if you have a high participation rate in a sample selected randomly from the population. In experiments, bias is also less likely if: subjects are randomly assigned to treatments; assignment is balanced in respect of any characteristics that might affect the outcome; subjects and researchers are blind to the identity of the active and control (placebo) treatments. Sample size is determined by acceptable precision of the estimate, expressed as the likely limits of the true value (95% confidence limits or interval). To halve the likely range you need 4x as many subjects.

Small sample sizes give acceptable accuracy for strong relationships Small sample sizes give acceptable accuracy for strong relationships. Therefore you can study batches of subjects until you get acceptable accuracy--sample size "on the fly". For an accurate estimate of a weak or trivial relationship between variables, a descriptive study usually needs a sample of hundreds or even thousands of subjects; a controlled trial usually needs scores of subjects; a crossover usually needs tens of subjects. In a descriptive study, the less valid the variable, the bigger the sample size. Validity--how well the observed value represents the true value--is often hard to determine; retest reliability (reproducibility of the observed value) sets an upper bound on validity and is worth determining in a pilot study. Lower validity of psychometric variables means ~4x as many subjects for adequate precision.

In an experiment, the less noisy--more reliable--the variable, the smaller the sample. Noise is represented by the typical (standard) error of measurement between trials. A crossover with ~8 subjects gives an outcome with a likely range of ± the typical error. A controlled trial needs ~32. If you have the time and resources, measure the typical error of the dependent variable(s) in a pilot study with the same time frame as the main study. Subjects differing in sex, age or other characteristics may differ in the relationship you are investigating. Therefore include these variables as covariates in the analysis. = multiple linear regression, analysis of covariance… In an experiment, subject characteristics or differences in training or other behavior before the treatment may explain individual responses to the treatment.

But you need bigger samples But you need bigger samples. For example, to estimate the difference in the effect of sex you will need 4x as many subjects (e.g., 40 females and 40 males instead of 20 males). So, if sample size is a problem, opt for a uniform sample of subjects (e.g., young females). In an experiment, try to measure variables that might explain the mechanism of the treatment. A putative mechanism variable has to at least partly track changes in the dependent variable, but that isn't sufficient to ensure it's a mechanism variable. Including such variables will increase your chances of publishing in a high-impact journal. In an unblinded experiment, such variables can help eliminate the possibility of a placebo effect. Important for PhD projects.