Research Designs Murray W. Enns Professor of Psychiatry

Broad Strokes The design is the structure of any investigation. It gives direction and systematizes the research. Each design has advantages and disadvantages. There are two main approaches to a research problem: Quantitative Research Qualitative Research

Qualitative research is a method of inquiry employed in many different academic disciplines particularly in the social sciences The form of data, by definition is not “quantitative measurement” and traditional statistics do not apply Strong theoretical foundation – e.g. Grounded Theory

Grounded theory A systematic methodology social sciences involving the construction of theory through the analysis of data. Begins with a question, or a collection of qualitative data. The data is reviewed to define repeated ideas, concepts or elements and tagged with codes, which have been extracted from the data. As more data is collected, codes are grouped into categories. These categories may become the basis for new theory.

Descriptive research Aim: Observe and describe E.gs. Case study Case series Surveys

The “lowly” case report Case reports have a time-honored and rich tradition in medical publication. Case reports can be relevant, timely, and important in advancing medical knowledge especially of rare phenomena. The observation of a single patient can add to our understanding of etiology, pathogenesis, natural history, and treatment, Excellent training opportunity for junior investigators

Appropriate use of case series data Proof (or Disproof) of concept for a new hypothesis Reporting of sentinel events Toxicities of therapies Recognition of epidemics Initial identification of previously unrecognized syndromes Studying outcomes of rare diseases or new treatments

Examples of case reports Four weeks after having performed the first human heart transplantation on December 3, 1967, Christian Barnaard published a case report in the South African Medical Journal In 1981, a series of case report publications noted unusual infections and neoplasms occurring in young men leading to the discovery of a new syndrome, AIDS

Reviewing other research Aim: Understand/explain what is known Literature review Systematic review Meta-analysis

Systematic review A systematic review can/should be a piece of true research done to establish whether a hypothesis is correct or not. It should begin with a structured protocol describing the aims and objectives, a clear search strategy, and an identified primary outcome Inclusion and exclusion of studies needs to be based on pre-determined criteria Accurate records - reflected in a flow diagram

Meta-analysis Meta analysis is a formal statistical method for aggregating the results of multiple studies Major advantage is the increased “N” which may increase power OR give better estimates of the sizes of effects Challenges: Cherry-picking versus systematic inclusion Garbage in, garbage out Publication biases – the file drawer effect

Funnel Plots Unbiased publications vs. Biased publications

PILOT study before full-scale Aim: to find out if the design works and is feasible Typical objectives of a pilot study: sample size calculation; a practice run of the protocol; testing data collection forms; testing randomization procedures; determining recruitment and consent rates; examining the acceptability of the intervention; selection of the most appropriate primary outcome measure.

When is a pilot study not a pilot study? Recent review: Shanyinde et al, 2011: Random sample of 50 “pilot” studies taken from MEDLINE 56% of papers- methodological issues discussed in substantial depth, 18% of papers- discussed future trials and only 12% of authors were actually conducting a future trial Conclusion: Sometimes a “pilot study” is just a confession of inconclusive results or lack of adequate sample size.

Correlational studies Aim: Predict Case-control study Observational study Longitudinal study Cross sectional study

Secondary Data Analysis This refers to the analysis of pre-existing data sets that were collected for another “primary” purpose Many large-scale MH epidemiological studies have had their data “liberated” for secondary analyses E.g. NCS-R, NEMESIS, NESARC, CCHS Huge benefits/efficiency with the “right questions” Major risk – inflation of risk of false positives due to multiple analyses being conducted E.g. the “post-hoc a-priori” hypothesis

Administrative Data Analysis Records collected for other administrative purposes (e.g. hospital records, physician billing records, DPIN) can be subject to analysis particularly if data from different sources is LINKED. *E.g. Manitoba Centre for Health Policy* 1.Health - Administrative, Clinical, Survey data 2. Education 3. Social - Healthy Child Manitoba, Community Services, Social Survey data 4. Justice

Experimental designs Aim: determine causes True experimental design e.g. Pre-test post-test Using a control group Randomization including RCTs Between subjects or within-subjects

SO, how do you choose? What information do you want? (aims of the study) The nature of the phenomenon – Is it feasible to collect the data? Is data already available elsewhere? How reliable does the information need to be? Is it ethical to conduct the study? The cost of the design Is there little or much current scientific theory and literature on the topic?