Week Three

 Key questions: ◦ Will there be an intervention? ◦ What specific design will be used?  Broad design options: ◦ Experimental (randomized control trial) ◦ Quasi-experimental (controlled trial without randomization) ◦ Nonexperimental (observational study)

 Key question: ◦ What type of comparisons will be made to illuminate relationships?  Some design options: ◦ Within-subjects design: Same people compared at different times or under different conditions ◦ Between-subjects design: Different people are compared (e.g., men and women)

 Control over confounds ◦ How will confounding variables be controlled? ◦ Which specific confounding variables will be controlled?

 Masking/Blinding ◦ From whom will critical information be withheld to avert bias?  Time frames ◦ How often will data be collected? ◦ When, relative to other events, will data be collected?

 Relative Timing ◦ When will information on independent and dependent variables be collected—looking forward or backward in time?  Location ◦ Where will the study take place?

 Many (if not most) quantitative research questions are about causes and effects.  Research questions that seek to illuminate causal relationships need to be addressed with appropriate designs.

Three key criteria for making causal inferences: The cause must precede the effect in time. There must be a demonstrated empirical relationship between the cause and the effect. The relationship between the presumed cause and effect cannot be explained by a third variable; another factor related to both the presumed cause and effect cannot be the “real” cause.

 Research designs vary in their ability to support the criteria for causal inference.  Experimental designs offer the strongest evidence of whether a cause (an intervention) results in an effect (a desired outcome). ◦ That’s why they are high on evidence hierarchies for questions about causes and effects.

 Manipulation: The researcher does something to some subjects—introduces an intervention (or treatment).  Control: The researcher introduces controls, including the use of a control group counterfactual.

 Randomization (also called random assignment): The researcher assigns subjects to groups at random. ◦ Typical assignment is to an experimental group or a control group. ◦ The purpose is to make the groups equal with regard to all other factors except receipt of the intervention.

 Involve an intervention but lack either randomization or control group  Two main categories of quasi-experimental designs: ◦ Nonequivalent control group designs  Those getting the intervention are compared with a nonrandomized comparison group. ◦ Within-subjects designs  One group is studied before and after the intervention.

 May be easier and more practical than true experiments, but ◦ They make it more difficult to infer causality. ◦ Usually there are several alternative rival hypotheses for results.

 If there is no intervention, the study is nonexperimental (observational).  Not all independent variables (“causes”) of interest to nurse researchers can be experimentally manipulated. ◦ For example, gender cannot ever be manipulated. ◦ Smoking cannot ethically be manipulated.

 Cause-probing questions (e.g., prognosis or harm/etiology questions) for which manipulation is not possible are typically addressed with a correlational design.  A correlation is an association between variables and can be detected through statistical analysis.

 In a retrospective correlational design, an outcome in the present (e.g., depression) is linked to a hypothesized cause occurring in the past (e.g., having had a miscarriage).  One retrospective design is a case–control design in which “cases” (e.g., those with lung cancer) are compared to “controls” (e.g., those without lung cancer) on prior potential causes (e.g., smoking habits).

 In a prospective correlational design, a potential cause in the present (e.g., experiencing vs. not experiencing a miscarriage) is linked to a hypothesized later outcome (e.g., depression 6 months later).  This is called a cohort study by medical researchers.  Prospective designs are stronger than retrospective designs in supporting causal inferences—but neither is as strong as experimental designs.

 Not all research is cause probing.  Some research is descriptive (e.g., ascertaining the prevalence of a health problem).  Other research is descriptive correlational— the purpose is to describe whether variables are related, without ascribing a cause-and- effect connection.

 Cross-sectional design—Data are collected at a single point in time.  Longitudinal design—Data are collected two or more times over an extended period. ◦ Trend studies ◦ Panel studies ◦ Follow-up studies Longitudinal designs are better at showing patterns of change and at clarifying whether a cause occurred before an effect (outcome).

 Controlling external factors ◦ Achieving constancy of conditions ◦ Control over environment, setting, time ◦ Control over intervention via a formal protocol  Controlling intrinsic factors ◦ Control over subject characteristics

 Randomization  Subjects as own controls (crossover design)  Homogeneity (restricting sample)  Matching  Statistical control (e.g., analysis of covariance)

 Statistical conclusion validity—the ability to detect true relationships statistically  Internal validity—the extent to which it can be inferred that the independent variable caused or influenced the dependent variable  External validity—the generalizability of the observed relationships across samples, settings, or time  Construct validity—the degree to which key constructs are adequately captured in the study

 Low statistical power (e.g., sample too small)  Weakly defined “cause”—independent variable not powerful  Unreliable implementation of a treatment— low intervention fidelity

Temporal ambiguity Selection threat — biases arising from pre- existing differences between groups being compared – This is the single biggest threat to studies that do not use an experimental design. History threat — other events co-occurring with causal factor that could also affect outcomes Maturation threat—processes that result simply from the passage of time Mortality threat—differential loss of participants from different groups – Typically a threat in experimental studies

 Inadequate sampling of study participants  Expectancy effect (Hawthorne effect) makes effects observed in a study unlikely to be replicated in real life.  Unfortunately, enhancing internal validity can sometimes have adverse effects on external validity.

 Research that integrates qualitative and quantitative data and strategies in a single study or coordinated set of studies  Advantages ◦ Complementarity—words and numbers, the two languages of human communication ◦ Incrementality—quicker feedback loops between hypothesis generation and testing ◦ Enhanced validity—triangulation strengthens the ability to make inferences

 Instrument development  Hypothesis generation and testing  Explication and illustration  Theory building and refinement  Intervention development

 Studies that involve an intervention: ◦ Clinical trials ◦ Evaluation research ◦ Nursing intervention research  Studies that do not involve an intervention ◦ Outcomes research ◦ Surveys ◦ Secondary analyses ◦ Methodologic research

 Studies that develop clinical interventions and test their efficacy and effectiveness  May be conducted in four phases

 Phase I: finalizes the intervention (includes efforts to determine dose, assess safety, strengthen the intervention)  Phase II: seeks preliminary evidence of effectiveness— a pilot test often using a quasi-experimental design  Phase III: fully tests the efficacy of the treatment via a randomized clinical trial (RCT), often in multiple sites; sometimes called an efficacy study  Phase IV: focuses on long-term consequences of the intervention and on generalizability; sometimes called an effectiveness study

 Emphasis on EBP has led to a call for studies that bridge the gap between tightly controlled efficacy studies and subsequent effectiveness studies.  Practical clinical trials (or pragmatic clinical trials) are designed to help in making decisions in real-world applications.

 Examines how well a specific program, practice, procedure, or policy is working  Clinical trials are sometimes evaluations of an intervention or program.  Some (but not all) evaluations are clinical trials because evaluations can address a variety of questions.

 Process (implementation) analysis  Outcome analysis  Impact analysis  Cost (economic) analysis

 Also called an implementation analysis  Yields descriptive information about how a program actually functions  Often combines qualitative and quantitative information

 Seeks preliminary evidence about program success  Common design: One-group pretest– posttest design

 Yields information about a program’s net effects  Typically uses an experimental or strong quasi-experimental design

 Also called an economic analysis  Assesses monetary consequences of a program—which may affect its ultimate viability  Typically done in connection with an impact analysis (or an RCT)

 Designed to document the quality and effectiveness of health care and nursing services  Often focuses on parts of a health care quality model developed by Donabedian; key concepts: ◦ Structure of care (e.g., nursing skill mix) ◦ Processes (e.g., clinical decision-making) ◦ Outcomes (end results of patient care)

 Typically relies on nonexperimental (correlational) designs  Tools include classification systems and taxonomies ◦ Nursing actions and diagnoses (e.g., NANDA) ◦ Nursing interventions (e.g., NIC) ◦ Nurse-sensitive outcomes (e.g., NOC)

 Obtains information (via self-reports) on the prevalence, distribution, and interrelations of variables in a population  Secures information about people’s actions, intentions, knowledge, characteristics, opinions, and attitudes  Survey data are used in correlational studies.

 Modes of collecting survey data: ◦ Personal (face-to-face) interviews ◦ Telephone interviews ◦ Self-administered questionnaires  Distributed by mail or the Internet  Personal interviews tend to yield the highest quality data but are very expensive.

 Advantages ◦ Researchers can collect extensive information fairly quickly. ◦ Can be used with many different populations ◦ Can be cross-sectional or longitudinal ◦ Questions limited only by what people are willing to answer  Limitations ◦ Data tend to be fairly superficial. ◦ Better for extensive than intensive inquiry

 Study that uses previously gathered data to address new questions  Can be undertaken with qualitative or quantitative data  Cost-effective; data collection is expensive and time-consuming  Secondary analyst may not be aware of data quality problems and typically faces “if only” issues (e.g., if only there was a measure of X in the dataset).

 Studies that focus on the ways of obtaining, organizing, and analyzing data  Can involve qualitative or quantitative data  Examples: ◦ Developing and testing a new data-collection instrument ◦ Testing the effectiveness of stipends in facilitating recruitment

 Population ◦ The aggregate of cases in which a researcher is interested  Sampling ◦ Selection of a portion of the population (a sample) to represent the entire population  Eligibility criteria ◦ The characteristics that define the population  Inclusion criteria  Exclusion criteria

 Strata ◦ Subpopulations of a population (e.g., male/female)  Target population ◦ The entire population of interest  Accessible population ◦ The portion of the target population that is accessible to the researcher, from which a sample is drawn

 Representative sample ◦ A sample whose key characteristics closely approximate those of the population—a sampling goal in quantitative research  More easily achieved with: ◦ Probability sampling ◦ Homogeneous populations ◦ Larger samples

 Sampling bias ◦ The systematic over- or under-representation of segments of the population on key variables when the sample is not representative  Sampling error ◦ Differences between sample values and population values

 Probability sampling ◦ Involves random selection of elements: each element has an equal, independent chance of being selected  Nonprobability sampling ◦ Does not involve selection of elements at random

 Convenience sampling  Snowball (network) sampling  Quota sampling  Purposive sampling

 Use of the most conveniently available people ◦ Most widely used approach by quantitative researchers ◦ Most vulnerable to sampling biases

 Referrals from other people already in a sample ◦ Used to identify people with distinctive characteristics ◦ Used by both quantitative and qualitative researchers

 Convenience sampling within specified strata of the population ◦ Enhances representativeness of sample ◦ Infrequently used, despite being a fairly easy method of enhancing representativeness

 Involves taking all of the people from an accessible population who meet the eligibility criteria over a specific time interval, or for a specified sample size ◦ A strong nonprobability approach for “rolling enrollment” type accessible populations ◦ Risk of bias low unless there are seasonal or temporal fluctuations

 Sample members are hand-picked by researcher to achieve certain goals ◦ Used more often by qualitative than quantitative researchers ◦ Can be used in quantitative studies to select experts or to achieve other goals

 Simple random sampling  Stratified random sampling  Cluster (multistage) sampling  Systematic sampling

 Uses a sampling frame – a list of all population elements  Involves random selection of elements from the sampling frame ◦ Not to be confused with random assignment to groups in experiments ◦ Cumbersome; not used in large, national surveys

 Population is first divided into strata, then random selection is done from the stratified sampling frames  Enhances representativeness ◦ Can sample proportionately or disproportionately from the strata

 Successive random sampling of units from larger to smaller units (e.g., states, then zip codes, then households) ◦ Widely used in national surveys ◦ Larger sampling error than in simple random sampling, but more efficient

 The number of study participants in the final sample ◦ Sample size adequacy is a key determinant of sample quality in quantitative research. ◦ Sample size needs can and should be estimated through power analysis.