Medical Research: Validity, Bias, Confounds, Probability, P-value, and Power Kimberly Pendell

Validity Validity implies that the measurement reliably measures what it intends to measure. Validity relates to the magnitude of bias. Example of non-valid measure: eye-color as measure of the likelihood of having a stroke Example of valid measure: diagnosis of atherosclerosis as measure of the likelihood of having a stroke

Bias Bias is the systematic tendency to produce an outcome that differs from the underlying truth. Bias is different from random error. Bias leads to systematic deviations (the error has direction). Random error is inherent in all measurement and is by chance--the average value of a measurement of a sample population is subject to variation from the true value for the entire population.

Some different types of bias Selection bias Interviewer bias Recall bias Detection bias Verification bias Publication bias Selection: An error in choosing the individuals or groups to take part in a study. Ideally, the subjects in a study should be very similar to one another and to the larger population from which they are drawn (for example, all individuals with the same disease or condition). If there are important differences, the results of the study may not be valid. Interviewer: Greater probing by an interviewer in one of two groups being compared Recall: patients who experience an adverse outcome have a greater likelihood of recalling an exposure than patients who do not have an adverse outcome, regardless of the true extent of exposure Detection (or surveillance): the tendency to look more carefully for an outcome in one of two groups being compared Verification: results of diagnostic test influence whether patients are assigned to treatment group Publication: occurs when the publication of research depends on the direction of the study results and whether they are statistically significant.

Bias and Research Designs Ex. research design Ex. of potential bias Ex. of potential solution RCT Selection bias Randomization Case-control study Recall bias Blind subjects to research hypothesis Meta-analysis Publication bias Look for unpublished data

Confounds A confound is a variable that is distributed differently in the study group and the control group and that affects the outcome being assessed. (Riegelman, 2005) Confounding variables can appear through bias or random chance. Find example!!

Probability Probability is the quantitative estimate of the likelihood of condition existing (as in a diagnosis) or of subsequent events (such as in a treatment study). (Guyatt, Rennie, 2002)

P-value P-value is the probability of an outcome occurring by chance. The p-value establishes statistical significance. In medical research the standard p-value is p<.05

Power Power is the ability of a research study to demonstrate statistical significance. Researchers determine the size of study population needed for a set amount of power before beginning their study. If the study has statistically significant results, the study has enough power. In medical research the standard for power is 80%.

Readings: Guyatt, G., Rennie, D. (Eds.). (2002). Users’ guide to the medical literature: a manual for evidence-based practice. USA: American Medical Association. Katzer, J., Cook, K.H., Crouch, W.W. (1998). Evaluating information: a guide for users of social science research. Massachusetts: McGraw Hill. Riegelman, R.K. (2005). Studying a study & testing a test: how to read the medical evidence. Philadelphia, PA: Lippincott Williams & Wilkins.